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EXCITED RANDOM WALKS WITH MARKOVIAN COOKIE STACKS 


ELENA KOSYGINA AND JONATHON PETERSON 


Abstract. We consider a nearest-neighbor random walk on Z whose probability oJx(j) to jump 
to the right from site x depends not only on x but also on the number of prior visits j to x. The 
collection {uJx(j))xe 2 ,,j>i is sometimes called the “cookie environment” due to the following informal 
interpretation. Upon each visit to a site the walker eats a cookie from the cookie stack at that site 
and chooses the transition probabilities according to the “strength” of the cookie eaten. We assume 
that the cookie stacks are i.i.d. and that the cookie “strengths” within the stack at site 

X follow a finite state Markov chain. Thus, the environment at each site is dynamic, but it evolves 
according to the local time of the walk at each site rather than the original random walk time. 

The model admits two different regimes, critical or non-critical, depending on whether the ex¬ 
pected probability to jump to the right (or left) under the invariant measure for the Markov chain 
is equal to 1/2 or not. We show that in the non-critical regime the walk is always transient, has 
non-zero linear speed, and satisfies the classical central limit theorem. The critical regime allows for 
a much more diverse behavior. We give necessary and sufficient conditions for recurrence/transience 
and ballisticity of the walk in the critical regime as well as a complete characterization of limit laws 
under the averaged measure in the transient case. 

The setting considered in this paper generalizes the previously studied model with periodic 
cookie stacks |KOS14 | . Our results on ballisticity and limit theorems are new even for the periodic 
model. 


1. Introduction 

Excited random walks (ERWs, also called cookie random walks) are a model for a self-inter acting 
random motion where the self-interaction is such that the transition probabilities for the next step 
of the random walk depend on the local time at the current location. To make this precise, a cookie 
environment a; = is an element of II = [0,1]^^^, and given a cookie environment 

a; G n and x G Z, the ERW in the cookie environment u started at x is a stochastic process 
{i^n}n>o with law such that P^{Xq = x) = 1 and 

P^ {Xn+I = + 11 Ao, Ai,..., A„) = 1 - (A ,+1 = A„ - 11 Ao, Ai,..., A,) 

= ^xM{k<n-. Xk = Xn}). 

The “cookie” terminology in the above description of ERWs comes from the following interpretation. 
One imagines a stack of cookies at each site x G Z. Upon the j-th visit to the site x the random 
walker eats the j-th cookie at that site which induces a step to the right with probability coxij) 
and to the left with probability 1 — LOx{j)- For this reason we will refer to cvxij) as the strength of 
the j-th cookie at site x. 

Many results for ERWs in one dimension were obtained under the assumption that there is 
an M < oo such that oJxij) = 1/2 for all j > M and x G Z |BW03l IMPV061 IBSOSal IBSOSbl 
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IKZ081 IKMlli IDK121 IPetl2l IPetlSl IA0161 IPetlSj . We will refer to this as the case of “boundedly 
many cookies per site”. Much less is known if the number of cookies per site is not bounded (in 
particular, if there are infinitely many cookies at each site). Notable exceptions are that Zerner 
|Zer05j proved a criterion for recurrence/transience and Dolgopyat |Dolll] proved the scaling limits 
for the recurrent case under the assumption that cox(j) > 1/2 for all x G Z and j > 1 (i.e., all 
cookies have non-negative drift). For a review (prior to 2012) of ERWs in one and more dimensions 
see |KZ13] . 

Until recently, very little was known for ERWs which had infinitely many cookies per site with 
both positive and negative drift cookiesj^ However, |KOS14] gave an explicit criterion for recur¬ 
rence/transience of ERWs in cookie environments such that the cookie sequence {u}x{j)}j>i is the 
same for all x and is periodic in j. The results in the current paper cover more general cookie envi¬ 
ronments where the cookie stack at each site is given by a finite state Markov chain. Moreover, we 
add to the results of [KOS14| by also proving a criterion for ballisticity (non-zero limiting velocity) 
and limiting distributions in the transient case. We also note that our model is general enough to 
include the case of periodic cookie stacks as in |KOS14) as well as some cases of boundedly many 
cookies per site. 


1.1. Description of the model. Let TZ = {1, 2,..., A^} denote a finite state space, and let {Rj}j>i 
be a Markov chain on TZ with transition probabilities given by an x matrix K = 

We will assume that this Markov chain has a unique closed irreducible subset TZq C TZ. Thus, 
there is a unique stationary distribution fj. for the Markov chain {Rj}j>i, and jjL is supported on 
TZq. Throughout the paper we will often represent probability distributions ry on 7^ as row vectors 
T] = ( 77 ( 1 ), ry(2),... ,r]{N)), so that for the stationary distribution /x we have fiK = /x. 

Assumptions (i.i.d., elliptic, Markovian cookie stacks). Let {R^}x£Z be an i.i.d. family of Markov 
chains R* = {Rj}j>i with transition matrix K. We assume that the cookie environments is given 
by oJxij) = p{Rj) for some fixed function p : TZ ^ {0,1). The requirement that p is strictly between 
0 and 1 will be referred to as ellipticity throughout the paper. 


For any distribution ry on TZ, let denote the distribution of {Rj}xez,j>i when each Rf, x G Z, 
has distribution ry. With a slight abuse of notation will also be used for the induced distribution 
on environments s which is constructed as in the assumptions above. If ry is concentrated on a 
single state i £ TZ, i.e. if ry = 6 i, we shall use instead of P^^. Expectations with respect to P,, or 
Pj will be denoted by and Ej, respectively. 

As stated above, the law of the ERW in a fixed cookie environment s is denoted Pf. We will refer 
to this as the quenched law. If the cookie environment has distribution P,^, then we can also define 
the averaged law of the ERW by P^{-) = E^[P^(-)]. Expectations with respect to the quenched 
and averaged measures will be denoted by and respectively. Again, if ry is concentrated on 
i GTZwe shall set Pf (•) := Ei[Pf{-)]. We will often be interested only in ERWs started at Xq = 0, 
and so we will use the notation P^^, P,^, or Pj in place of P®, P° or P°, respectively. 

Example 1.1 (Periodic cookie sequences). The model above clearly generalizes the case of periodic 
cookie sequences at each site as in |KOS14| . To obtain periodic cookie sequences set = 1 for 

r = 1,..., A^ — 1, K]\fp = 1, and let rj = 61 . 


Example 1.2 (Cookies stacks of geometric height). Fix a G (0,1], p(l) > 1/2, and p{2) = 1/2. 

( 1 — (y Q: \ 

Q ^ 1. If ry = (5i then the cookie environment is such that there are an i.i.d. 
Geometric(a) number of cookies of strength p{l) > 1/2 at each site. 


^With the exception of one-dimensional random walk in random environment which can be seen as a particular 
case of ERW where 0Jx{j) = u;a;(l) for all j > 1 at each x G Z. 
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Example 1.3 (Bounded cookie stacks). The model above also generalizes the case of finitely many 
cookies per site. For instance, let the transition matrix K be such that = 1 for i = 1,, N—1 

and = 1, and let the function p : 7^ —)■ (0,1) be such that p{N) = 1/2. In this case, if r/ = di 

then ujx{j) = p{j) if / < — 1 and p{j) = 1/2 if j > N. With a little more thought, one can also 

obtain random cookie environments that are i.i.d. spatially with a bounded number of cookies at 
each site as in |KZ08j subject to the restrictions that there are only finitely many possible values 
for the cookie strengths iOx{j) and that ujx{j) G (0,1). For instance, if 


K = 


/O 1 0 0 0\ 

0 0 0 0 1 

0 0 0 1 0 

0 0 0 0 1 

\0 0 0 0 1 / 


p = 


p{2) 

p{3) 

p(4) 

VV2/ 


and r/= (a, 0,1 — a, 0,0), 


then the corresponding cookie environments under the distribution have 2 cookies at each site, 
having stack (p(l),p(2)) with probability a and stack (p(3),p(4)) with probability \ — a. 


Before stating our main results, we note two basic facts for ERWs that are known to hold in 
much more generality than our model. We will use these facts as needed throughout the paper. 


Theorem 1.4 f [ABOIB) Theorem 1.2] and |KZ13t Theorem 4.1]). 

(i) Zero-one law for transience. For any initial distribution rj on TZ 

Pn ( lim Xn = oo) , Pr, { lim Xn = —oo) G { 0 , 1 }. 

(ii) Strong law of large numbers: For any initial distribution tj on TZ there is a determin¬ 
istic Vo G [—1; 1] such that 


Pr, ( lim Xn/n = uo ) 
\n—>-oo J 


= 1 . 


1.2. Main results. The function p : 7^ —)■ (0,1) used in the construction of the cookie environment 
w corresponds to a column vector p = (p(l),p(2),... ,p{N)Y with i-th entry given hy p{i). Note that 
our assumptions on the Markov chains {R^}x£Z: imply that the limiting average cookie strength 
at each site is 


lim j'^ujxij) = fi-p=:p, 




and that this limit does not depend on the distribution g of lVx{1). It is natural to suspect that 
the ERW is transient to the right (resp. left) when p > lj2 (resp. p < 1/2). Our first main result 
below confirms this is the case. Moreover, we also establish that if p 7 ^ 1/2 the walk has non-zero 
linear speed and a Gaussian limiting distribution. 


Theorem 1.5. Assume that p 7 ^ 1 / 2 . 

Transience: If p > 1/2 then P^(lim„^oo-Tn = 00 ) = 1 for any initial distribution g on TZ. 

Similarly, if p < 1/2 then P^(lim„_,.oo = — 00 ) = 1 for any g. 

Ballisticity: For any initial distribution g on TZ the limiting velocity vq (see Theorem 1. 

is non-zero: it is positive for p > 1/2 and negative for p < 1/2. 

Gaussian limit: For any distribution g on TZ there exists a constant b = b{K, p,g) >0 
such that 


lim Pjj 
n^oo ^ 


Xn - nvo 


b^/h 



<h(x) 
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The case p = 1/2 is more interesting (we will refer to this as the critical case). In fact, it will 
follow from our results below that ERWs in the critical case p = 1/2 can exhibit the full range of 
behaviors that are known for ERWs with boundedly many cookies per site (e.g., transience with 
sublinear speed and non-Gaussian limiting distributions). Before stating our results for the critical 
case, we remark that in the case of boundedly many cookies per site the classification of the long 
term behavior of the ERW is based on a single parameter 5 = “ 1)] which is the 

expected total drift contained in a cookie stack. Our results below depend on two parameters 5 and 
6 which are also explicit but do not admit such simple interpretation. However, if we denote by Tq 
the number of upcrossings (steps to the right) the ERW makes from 0 before the n-th downcrossing 
(step to the left) from 0 then 

2n Var IT”! 

6 = 5(ri, K,p) :=—, where p := lim (EnlTn] — n), and u := lim - - — 

U n^oo n^oo 77, 


The parameter 6 is defined in a symmetric way using downcrossings instead of upcrossings and vice 
versa so that 6 := 6 ( 7 ], K,1 — p). The explicit formulas for the parameters are not intuitive and 
are given in (37) and (pSl). In the special case of boundedly many cookies per site the parameter 


5 agrees with the one used in previous papers, 5 = —<5, and v = 2. Proposition 4.3 below shows 
that in general d + J = \ — 2/v where v need not be equal to 2 (see Example 1.13), and so two 
parameters are needed in the general model. 


Theorem 1.6. Fix an arbitrary initial distribution rj on TZ and let p = p-p = 1/2. Then, there 
exist constants 6 and 6 satisfying 5 + 5 < 1 (see (37) and (38) for the explicit formulas in terms of 
T], K, and p/ and such that 

• if 6 > 1, then Pr,(lim„^oo = 00 ) = 1; 

• if 6 > 1, then P^(lim„^oo= — 00 ) = 1; 


if 6 < 1 and (5 < 1 then Rj(liminf„_,.oo Xn = — 00 , limsup. 


n—>00 Xn 


= 00 ) = 1 . 


The next theorem characterizes exactly when the walk is ballistic (i.e., has non-zero limiting 
velocity). 


Theorem 1.7. Fix an arbitrary initial distribution rj on TZ. Let p = 1/2, 5 and 5 be as in Theorem 


• vq > 0 if and only if 6 > 2 . 

• vq < 0 if and only if 6 > 2 . 


1.6, and vq be the limiting velocity (see Theorem [7/ ][ii)^. Then 


Our final main result concerns the limiting distributions in the transient cases. We will state 
the results below for walks that are transient to the right {6 > 1 ), though obvious symmetry 
considerations give similar limiting distributions for walks that are transient to the left (5 > 1 ) 
by replacing 5 with 6 . The theorem below not only gives limiting distributions for the position 
Xn of the ERW, but also for the hitting times T„, where for any x G Z the hitting time T^ = 
inf{A: > 0 ; Xk = x}. Eor a G (0, 2) and 6 > 0 we will use Lafi{-) to denote the distribution of the 
totally-skewed to the right a-stable distribution with characteristic exponent 


( 1 ) 


log [ Lo,b(fix) 

JR 


— 6 |u|“ (1 — itan(™) sign(u)) 07 ^! 
— 6 |u| ( 1 -b ^ log |u| sign(u)) a = 1 . 


Also, as in Theorem 1.5 we will use 4>(-) to denote the distribution function of a standard normal 
random variable. 


Theorem 1.8. Let p = 1/2 and suppose that <5 > 1. 
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(i) If S £ (1,2), then there exists a constant b> 0 such that 

A < *) = ii/2,i(*) F, = 1 - 

(ii) If 5 = 2, then there exist constants a,b > 0 and sequences D{n) ~ a“^logn and r(n) 
an/log(n) such that 


lim P, 


Tn - nD{n) 


n 


< x \ = Li^i){x) and lim 


Xn - r(n) 
n/(logn)2 


< X = 1 - Lifi{-x/a^). 


(iii) If 6 £ (2,4), then there exists a constant b > 0 such that 


lim P, 


Tn - n/vQ 


n 


2/5 


and 


Xn - nVQ 
2/5 


< X = 1 — L 


S/2,b( — X)- 


(iv) If 6 = 4, then there exists a constant b > 0 such that 


lim P, 


Tn - n/vQ 
by/ n log n 


< X ) = ^{x) and lim 


Xn - nVQ 
bv^'^y/n logn 


< X = $(x). 


(v) If 6 > 4, then there exists a constant b > 0 such that 


lim Pr 


Tn - n/vp 

by/n 


< X ) = $(x) and lim 


Xn - nVQ 
bvQ^'^y/n 


< X = 4>(x). 


1.3. An overview of ideas and open qnestions. Our approach is based on the following three 
ingredients which appeared in the literature 65-20 years ago and were successfully used by other 
authors in a number of different contexts: (1) mappings between random walk paths and branching 
process trees, (2) diffusion approximations of branching-like processes (BLPs) and Ray-Knight 
theorems, and (3) embedded renewal structures for BLPs. 

(1) Mappings between random walk paths and branching process trees. The existence of a bijection 
between excursions of nearest-neighbor paths on Z and rooted trees is well known and was observed 
as early as |Har52| Section 6]. In this bijection, if the rooted tree is viewed as a genealogical tree 
then the offspring of the individuals in the n-th generation corresponds to the number of times the 
nearest-neighbor path on Z steps from n to n-|-1 in between steps from n to n — 1. Properties of the 
random walk can then be deduced from properties of the corresponding random trees. For instance, 
the maximum distance of the excursion from 0 corresponds to the lifetime of the branching process, 
and the return time of the walk to the origin is equal to twice the total progeny of the branching 
process over its lifetime. 

Similarly, a related bijection is known to exist between rooted trees and nearest-neighbor paths 
in Z from 0 to n. In this bijection, the rooted tree corresponds to a genealogical tree where there 
is one immigrant per generation (for the first n generations only) and the offspring of individuals 
in the f-th generation correspond to the number of steps from re — / ton — i — 1 between steps from 
n — iton — i + 1. As with the first bijection, this bijection also can be used to deduce properties 
of a random walk from properties of the corresponding random tree. For instance, the hitting time 
Tn = inf{A: > 1 : X^ = re} is equal to re (the total number of immigrants) plus twice the total 
number of progeny of the branching process over its lifetime. 

For either of the above bijections, putting a probability measure on random walk paths induces a 
probability measure on trees and vice versa. For instance, for simple symmetric random walks on Z, 
the first bijection gives a measure on trees corresponding to a critical Galton-Watson process with 
Geometric(l/2) offspring distribution, and the second bijection corresponds to a Galton-Watson 
process with Geometric(l/2) offspring distribution but with an additional immigrant in the first 
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n generations. These bijections were at the core of F. Knight’s proof of the classical Ray-Knight 
theorem in |Kni63j . For one-dimensional random walks in a random environment (RWRE), the cor¬ 
responding measures on trees are instead branching processes with random offspring distributions 
amd the second bijection above was used in |KKS75j to obtain limiting distributions for transient 
one-dimensional RWRE (under the averaged measure). The advantage of using this bijection to 
study the RWRE is that while the random walk is non-Markovian under the averaged measure, 
the corresponding branching process with random offspring distribution is a Markov chain. Subse¬ 
quently, it has also been observed that several other models of self-interacting random walks also 
have this Markovian property for the BLPs which come from the above bijections. In particular, this 
applies to a large number of self-interacting random walk models where the transition probabilities 
for the walk depend on the past behavior of the walk at the present site. Included in this are several 
models of self-repelling/attracting random walks [T6t95l IT6t96l IPinlOl IPT16] as well as excited 
random walks with boundedly many cookies per site. For ERWs, this branching process approach 
was first used in [BSn8a| and has since been the basis for many of the subsequent results on ERWs; 
see for instance the review |KZ13j as well as more recent works in |KZ141 IKOS141IA0161 IPetl5] . 

(2) Diffusion approximations of BLPs and Ray-Knight theorems. Diffusion approximations of 
branching processes is an extensively studied topic, especially in the context of applications to 
population dynamics (see, for example, books |Jag75| , |Kur81| ). The fact that a rescaled critical 
Galton-Watson process converges weakly to a diffusion was first observed in [Fel51| and then rig¬ 
orously proven in |.Iif69] . In particular, a Galton-Watson process with Geometric(l/2) offspring 
distribution converges to one-half of a zero-dimensional squared Bessel process. Adding an immi¬ 
grant in each generation raises the above dimension to two so that the limiting process becomes 
one-half of the square of a standard two-dimensional Brownian motion. Recalling the connection 
of simple symmetric random walks with critical Galton-Watson processes from the bijections given 
above and the fact that Brownian motion is the scaling limit of random walks, we can think about 
the first and second classical Ray-Knight Theorems as continuous versions of these bijectionsj^ 

For a class of non-Markovian self-interacting random motions, a generalized Ray-Knight theory 
was developed by Balint Toth (see |T6t99| and references therein). For these walks the corre¬ 
sponding diffusion process limits for the local times are squared Bessel processes of appropriate 
dimensions. Using this generalized Ray-Knight theory the author obtains limiting distributions for 
the random walk stopped at an independent geometric time. Additionally, for a certain sub-class 
of these self-interacting random walks he identifies these limiting distributions as those of a Brow¬ 
nian motion perturbed at extrema |T6t96[ Remark on p. 1334] and asks if the observed connection 
extends to multi-dimensional distributions. 

Even though the dynamics of ERWs are quite different from those of the self-inter acting random 
walks in |T6t99j . the corresponding rescaled BLPs also converge to squared Bessel processes (see 
|KM11| . [KZ14| . [DK15| . and Lemmas |6.1| and [TT] below). Similarly to |T6t96| these diffusion limits 
can be used to infer certain properties of the scaling behavior of the ERW, though alone they are 
not quite sufficient to obtain limiting distributions for the ERWs. 

(3) Embedded renewal structure. To obtain limit theorems for transient ERWs we essentially 
follow an outline which was first presented in [KKS751 pp. 148-150] in the context of transient one¬ 
dimensional random walks in random environments. Starting with [BSOSbj . this strategy has been 
used in essentially all papers concerned with limit laws for transient one-dimensional ERW. The 
idea is to first use the bijection above which relates the hitting time to n plus twice the total 
progeny of a BLP, and then to compare the total progeny of this BLP to a sum of i.i.d. random 

^The second Ray-Knight theore m was exten ded from Brownian motion to a large class of symmetric (and some 
non-symmetric) Markov processes f |EKM'*~00| i. These extensions found many new applications. The interested 
reader is referred to books |MR.06] . |Sznl2] and references therein. 
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variables using regeneration times of the BLP. To obtain limiting distributions for the hitting times 
(and then also the position) of the ERW using this approach, the key is to obtain precise tail 
asymptotics of both the regeneration times and the total progeny between regeneration times of 
the BLPs. 

In [BS08b| and [KZ08] . the necessary tail asymptotics for the BLPs were obtained using gener¬ 
ating functions or modifications of the process which allowed for the application of known results 
from the literature for branching processes with migration. The approach based on squared Bessel 
diffusion limits of the BLPs and calculations in the spirit of gambler’s ruin was proposed in |KM11) 
and developed in subsequent papers |KZ14) , |DK15] . This approach not only eliminates the need to 
quote results from the branching process literature but also allows one to obtain new results about 
BLPs. In the current work we push the method further by lifting the limitation on the number 
of cookies per site at the cost of requiring a Markovian structure within the cookie stacks. We 


provide the required tail asymptotics in the full critical regime (Theorems 2.6 and 2.7), show the 
one-dimensional limit laws in the transient case, but leave the recurrent case and functional limit 
theorems for future work. 


1.3.1. Comments and open questions. 


(i) Using the formulas in (37) and (38) below, one can explicitly calculate the parameters 5 and 5 


in terms of rj, K, and p. Therefore, recurrence/transience, ballisticity, and the type of the limiting 
distribution can be determined for any given example. However, there is no explicit formula for the 


limiting velocity vq when vq 0 or for the scaling parameters a, 6 > 0 that appear in Theorem 1.8 


Question 1.9. What can he said about monotonicity and strict monotonicity of vq with respect to 
the cookie environment^ 


(ii) Theorems 1.7 and |1.8| give the first such results for ERWs with an unbounded number of 


cookies per site (with the exception of the special case of random walks in random environments). 


As mentioned above, the recurrence/transience results in Theorem 1.6 were known for excited 


random walks with unbounded number of cookies per site only in the special cases of non-negative 
cookie drifts {oJx{j) >1/2 for all j) |Zern5| or periodic cookie stacks |KOS14| . In |KOS14] the 
authors also use a BLP, but their proof of the criterion for recurrence/transience differs from ours 
and is based on the construction of appropriate Lyapunov functions rather than on the analysis 
of the extinction times of the BLP. It is possible that their proof may be extended to include our 
model, but such an extension is not automatic since it requires an additional strong concentration 
estimate (see |KOS14l Theorem 1.3]) while our approach seems much less demanding. 

(hi) Eunctional limit theorems have been obtained for ERWs with bounded cookie stacks under 
the i.i.d. and (weak) ellipticity assumptions. The transient case was handled in [KZ081 Theorem 3] 
and |KZ13l Theorems 6.6 and 6.7]. Scaling limits for the recurrent case were obtained in |Dolll) 
and |DK12j . Excursions from the origin and occupation times of the left and right semi-axes for 
ERW with bounded number of cookies per site have also been studied |KZ14j . |DK15j . We believe 
that similar results hold for the model considered in the current paper and leave this study for 
future work. 

Question 1.10. State and prove functional limit theorems for the critical transient case (p = 1/2, 
maxjd, 5} > 1/. 

Question 1.11. Study scaling limits and occupation times of the right and left semi-axes for the 
recurrent case (p = 1/2, maxjd, <5} <1/. 


^See |Petl3 | and |HoI15| for the up to date account of the known results. 
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Question 1.12. Show that the critical ERW (i.e. p = l/2j is strongly transient under Py^ if and 
only f/max{5, (5} > 3. Show that the non-critical ERW (i.e. p 7 ^ l/2j is always strongly transient. 

1.4. Examples. In this subsection we give some examples where the parameters 5 and 6 are 


explicitly calculated using the formulas in (37) and (38). The calculations are somewhat tedious 


to do by hand, but since the formulas are explicit one can usually compute the parameters very 
quickly with technology. 

Example 1.13 (Periodic cookie stacks). In the setting of Example [13 


Ef=iEii(i-p(j))(2p(*)-i) 


5 = 


Ef=iELiP(j)(i-2p(i)) 


2 Ef=ip(j)(i - pU)) 2 P(j)(i -P(J)) 

For this model, the characterization of recurrence and transience was proved previously in |KUS14j 
but the results in Theorems 1 1. 71 and 11.81 are new. 


and S = 




Example 1.14 (Cookie stacks of geometric height). In the setting of Example 1.2 one obtains that 
6 = —6 = {2p{l) — l)/a. Note that Ei[^^j^(2a;o(j) —1)] = (2p(l) —l)/a as well; this agrees with the 
criteria for recurrence/transience for this example that can be obtained from [Zern5] . Previously, 
there were no known results regarding ballisticity or limiting distributions for this example. 



P 


Figure 1. A plot showing the regions for different types of behavior occurring for 


cookie environments as in Example 1.14 


Example 1.15 (Two-type, critical). Let K = 


1 — a a 
a 1 — a 


for some a G (0, 1) and let p = 


{p, 1 — pY for some p > 1/2. In this case, if we use the initial condition r] = (1,0) then a calculation 
done with Mathematica yields 


( 2 ) 


6 = 5 


1 — a a 
a. 1 — a 


P 
l-p 


( 1 , 0 ) = 


i2 p-l){{2a-l)p-a) 


4(2q! — 1){p — l)p -|- a — 1 ’ 

Note that if a G (0,1/4) then the parameter 6 is non-monotone in p. In fact, for a < ^^" 55 ^ ~ 
0.00450487 the parameter 5 starts at 0, increases to a value larger than 4, and then decreases to 1 
as p ranges from 1/2 to 1 (See Figure]^. 


„>o is said to be strongly transient under P,, if it is transient and Erj[R \ R < 00 ] < 00 , where R ~ inf{n > 
1 : = Xo}. 
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0.5 0.6 0.7 0.8 0.9 1.0 



Figure 2. The left plot shows the regions for different types of behavior occurring 
for cookie environments as in Example |1. 15 The right plot shows the value of S as 
a function of p G ( 1 / 2 , 1 ) when a = 0 . 002 . 


Note also that the case a = 0 corresponds to a classical simple random walk which steps to the 
right with probability p > 1/2 on each step. However, taking a = 0 in Q gives d = 2 ^^ and thus 
the results of Theorems 1 1. 7| and 1 1. 8 | clearly do not hold when a = 0 (this is not a contradiction since 
the transition matrix K is the identity matrix when a = 0 and there is more than one irreducible 


closed set). The case a = 1 gives periodic cookie stacks with period 2 and the 
which agrees with the formula obtained in |KOS14| . see Example 


X _ (2p-l)(l-p) 
4p(l-p) 


brmu la Q gives 


1.13 


1.5. Structure of the paper. In Section we introduce our main tool, forward and backward 


BLP, and show how the main results in the critical case (Theorems 1.6 - 1.8) can be deduced 


from Theorems 2.6 and 2.7 about the behavior of the tails of the lifetime and the total progeny 
over a lifetime of these BLP. Much of the remainder of the paper therefore is focused on the 

Section discusses the 


27l\ 


4 ] extends the results of the 


analysis of these BLPs, with the goal of proving Theorems 2.6 and 
asymptotics of the mean and variance of the forward BLP. Section 
previous section to the backward BLP and establishes key relationships between parameters of the 
BLPs; in particular, in Section we give explicit formulas for 6 and 6 . In Section we treat the 
non-critical case (p 7 ^ 1/2) deriving Theorem 1.5 Using the formulas for the parameters of the 


BLPs derived in the previous sections, we show how the BLPs in the non-critical case p > 1/2 can 
be coupled with critical BLPs for which the parameter 5 is arbitrarily large. From this coupling 
we then show that the conclusions of Theorem lL5] can be derived from Theorems 12.61 and 12.71 in 
the same way as the proofs of Theorems 1.6-1.8 were obtained when J > 4. Finally, in Sections 


and we discuss proofs of Theorems M and |2.7[ The crucial step here is showing that the BLPs 
have scaling limits that are squared Bessel processes. The generalized dimensions of these squared 
Bessel processes depend on the asymptotics of the mean and variance of the BLPs computed in 
Sections and This provides the connection of the parameters 6 and 6 dehned in Section with 


the tail asymptotics exponents in Theorems 2.6 and 2.7 


2. The associated branching-like processes 

In this section, we introduce two BLPs that are naturally associated with the ERW and will 
prove the main theorems for the critical case assuming the necessary results about these BLP. 
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Given a cookie environment ui, we will expand the measure Pt^ to include an independent family 
of Bernoulli random variables such that ^x(j) ~ Bernoulli(t<;x(j))- The ERW can 

then be constructed from the ^x{j) as follows: if Xn = x and X]fc=o ^{Xk=x} = P then X^+i = 

~ 1 - 

Remark 2.1. The arrow systems construction of the ERW introduced by Holmes and Salisbury 
|HS12] is very similar to the above coin toss construction: compare the last equation with [HollSl 
p. 2, above Theorem 1.1]. The distinction lies in the emphasis of the arrow approach on combina¬ 
torial results obtained by coupling of arrow systems rather than probability measures. 

We now show how the family of Bernoulli random variables {^xO)} can also be used to construct 
the associated forward and backward BLP. 

2.1. The forward branching-like process. The excursions of a random walk to the right of the 
origin induce a natural tree-like structure on the right-directed edge local times of the walk. That 
is, for X > 1 jumps from x to x -|- 1 can be thought of “descendants” of previous jumps from x — 1 
to X. To be precise, if we set 70 = 0 and for n > 1 and x > 0 let 

7n-l 

7 n = inf{A: > 7 „_i : Xk-i > 0 and Xk = 0 } and = '^ '^{Xk=x,Xk+i=x+i}, 

k=Q 

then 7 „ is the time of the n-th return to the origin from the right and is the number of times 
the random walk has traversed the directed edge from x to x -t- 1 by time 7 „ (note that 7 ^ can be 
infinite if the walk is transient to the right or left). 

If the random walk makes n excursions to the right ( 7 ^ < 00 ), then the directed edge local times 
£^ can be computed from the Bernoulli random variables. If g^-i = 1 x 1 , then since the steps of 
the random walk are ±1 it follows that the walk makes m jumps to the left from x by time 7 ^. 
Thus, is the number of jumps that the random walk makes to the right from x before the m-th 
jump to the left. If we refer to a Bernoulli random variable ^x{j) as a “success” if ^x(j) = 1 and a 
“failure” if Cx(j) = 0, then £^ is the number of successes in the Bernoulli sequence ^x = {^x(j)}j>i 
before the m-th failure. More precisely, introducing the notation 

k>0: 

we have that if 7,1 < 00 and g^-i = rn then £^ = S^. Note that 5 q = 0 by the convention that 
an empty sum is equal to zero. Also, let so that is the number of successes 

between the (m — l)-st failure and the m-th failure in the Bernoulli sequence ^x- 

With the above directed edge process g^ as motivation, we define the forward BLP started at 
y > 1 by 

Ui-i 

(3) Uo = y and Ui = = '^G"^ for i > 1. 

m=l 

We will use the notation and P^'^{-) = {■)] for the quenched and averaged distri¬ 

butions, respectively, of the forward BLP started at y. Note that under the quenched measure 
P^’^ the forward BLP is a time-inhomogeneous Markov chain, but since the cookie environment 
is assumed to be spatially i.i.d. the forward BLP is a time-homogeneous Markov chain under the 
averaged measure p!/'^ for any initial distribution 7 . 

The following Lemma summarizes the connection of the forward BLP with the directed edge 
local times of the random walk. 
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Lemma 2.2. If Uq = n > 1, then Ui > for all i > 0. Moreover, on the event { 7 ^ < 00 } we 
have Ui = £f for all i >0 if Uq = n. 

The equality Ui = T” on the event { 7,1 < 00 } was described above. For the proof of the general 
inequality U > £f we refer the reader to |KZ08[ Section 4] or |Petl3l Lemma 2.1], 

Remark 2.3. Note that in the case of classical simple random walks (i.e., ujx{j) = p G (0,1)) 
{Gin} iez, m>i is a sequence of i.i.d. Geometric(l —p) random variables. In this case, it is clear from 
Q that Ui is a branching process with Geometric(l — p) offspring distribution. For ERWs with 
boundedly many cookies per site the 0]^, are i.i.d. Geometric(l/2) for all m sufficiently large and 
thus one can interpret Ui as a branching process with (random) migration (c.f. |KZ08j . or more 
explicitly but in the context of the backward BLP see |BS08a) i. However, in the more general setup 
of the current paper one can no longer interpret Ui as a branching process and so we simply refer 
to Ui as a “branching-like” process. 

Remark 2.4. One can also obtain a similar BLP which is related to the excursions of the ERW to 
the left of the origin. Clearly this BLP would have the same law as the forward BLP Ui defined 
here but with p replaced by 1 — p. 

2.2. The backward branching-like process. The backward BLP is related to the random walk 
through edge local times. However, the backward BLP is related to the local times of the left 
directed edges when the random walk first reaches a fixed point to the right of the origin. To be 
precise, 

T„-l 

Hn — l{A:fe=a:, Afc+i= 3 ;-l}) n > 1 , X < n, 
fc =0 

be the number of steps to the left from x before time Tn (recall that Tn = inf{fc > 0 ; Xk = n} is 
the hitting time of n by the ERW). On the event {Tn < oo}, the sequence of directed edge local 
times {T>*}a;<n also have a branching-like structure. Jumps to the left from x -|- 1 before time Tn 
give rise to subsequent jumps to the left from x before time Tn- However, one important difference 
with the forward BLP should be noted in that not all jumps to the left from x are “descendants” 
of jumps to the left from x -|- 1. In particular, for x > 0 the random walk can jump to the left from 
X before ever jumping from x to x -|- 1. 

As with £Jf above, the directed edge process P® can be computed from the Bernoulli random 
variables f,x{j)- In particular, if 

{ fc+m 

^ > 0 : ^ fxU) = m 

denotes the number of failures in the Bernoulli sequence ^x = {^a;(j)}j>i before the m-th success 
then it is easy to see that if < oo then P” = 0 and 

(4) if = m then Vf = J ° for x < n. 

Indeed, if x > 0 and there are m jumps from x + 1 to x before time Tn then there must be m -|- 1 
jumps from x to x -|-1 (the initial jump from x to x -|-1 plus m more jumps which can be paired with 
a prior jump from x -|- 1 to x). Thus, from the construction of the random walk via the Bernoulli 
random variables f,x{j) above, it follows that the number of jumps from x to x — 1 before time Tn is 
the number of failures before the (m -|- l)-th success in the Bernoulli sequence ^x- The explanation 
of @ when X < 0 is similar, with the exception that all jumps to the right from x can be paired 
with a prior jump to the left from x -|- 1 . 
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Again with the directed edge local time process as motivation, we define the backward BLP 
started at y > 0 by 

(5) Vb = y and Vi = Py._i+i for i > 1. 

We will use and to denote the quenched and averaged laws of the backward branching 
process started at y > 0. As with the forward BLP, 14 is a time-inhomogeneous Markov chain 
under the quenched measure and a time-homogeneous Markov chain under the averaged measure. 

Lemma 2.5. If n > 1 and Prj{Tn < oo) = 1, then the sequence (P”, ..., 2 ?^,has the 

same distribution under P^j as the sequence (Vb) ki; • • • j 14 - 1 ; Vn) under the measure P^'^. 

Proof. Porn > 1 let be the family of Bernoulli random variables given by {j ) = 

fn-x{j) for X G Z and j > 1, and let be the backward BLP started at = 0 but defined 
using the Bernoulli family in place of {^x(j)}- Then, it is clear from @ and © on the 

event {T„ < oo} that T’””* = foi’ * = 0,1,... ,n. Finally, since the cookie environments are 

spatially i.i.d., it follows that {f!'x^\j)'\x&,j>i and {f,x{3)}x&i, j>i have the same distribution under 
the averaged measure Prj = □ 


2.3. Proofs of the main results in the critical case. Here and throughout the remainder of 
the paper, we will use the following notation for hitting times of stochastic processes. If {Zj}j>o is 
a stochastic process, then for any x G M let and be the hitting times 

:= inf{j > 0 : Zj < x} and := inf{j > 0 : Zj > xj. 


Usually the stochastic process Z will be either the forward or backward BLP, though occasionally 
we will also use this notation for other processes. 

The analysis of the forward and backward BLP is key to the proofs of all the main results in this 
paper. In particular, all of the results in the critical case p = 1/2 (Theorems 1.6-1.8) will follow 
from the following two Theorems. 


Theorem 2.6. Let p 

then 


1/2 and let 6 be given by {38). 


If U is the forward BLP defined in 0, 


• If 5 > 1 then P^'^{aQ = oo) > 0 for all y > 1. 

* If d < 1 then for any y > 1 there are positive constants ci = ci(y, y) and C 2 = C 2 (y,y) 
such that 


( 6 ) 


lim ^ 

n^oo 


Pi'-n.rJ' > n) 


Cl 


and 


lim 

n^oo ' 



= C2, 


where for 5 = 1 we replace with Inn. 


Theorem 2.7. Let p 

then 


1/2 and let 6 be given by (38). If V is the backward BLP defined in 0, 


• If S < 0 then P^'^{aQ = oo) > 0 for all y > 0. 

• If d > 0 then for any y > 0 there are positive constants C 3 = C 3 (y, y) and C 4 = C 4 (y,y) 
such that 


(7) 



and 



= C 4 , 


where for 6 = 0 we replace with Inn. 
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Remark 2.8. Analogous theorems are known for the ERW with bounded number of cookies per 
site, |KM11[ Theorems 2.1, 2.2], |DK15[ Theorem 1.3]. 


We will give the proofs of Theorems 2.6 and|2.7|in Sectionsandbelow, but first we will show 


how they are used to prove Theorems 1.6-1.8 


Proof of Theorem M For any cookie environment a;, let a;"*" be the modified cookie environment 
in which = oJxif) for a: / 0 , j > 1 and with = 1 for all j > 1 (that is, in the cookie 

environment the walk steps to the right after every visit to the origin). Recall that 'jn is the 


time of the n-th return to the origin. Lemma 2.2 implies that 

( 8 ) Puj+iln < oo) = P^+{Sf = 0 , for all x sufficiently large) < P^’"'{o'q < oo). 

(Note that in the last probability on the right we can change the cookie environment from uj~^ to u 
since the forward branching process is generated using the Bernoulli random variables f,xij) with 
X > 1.) If instead 7 „ = oo, then since the coxif) are uniformly bounded away from 0 and 1 the walk 
cannot stay bounded for the first n excursions to the right. Therefore, 

(9) Puj+iln = oo) = P^+i£f > 1, Vx > 1) < = oo), 

where the last inequality follows from Lemma |2.2[ Combining ([^ and Q we can conclude that 

( 10 ) 


Puj+iln < oo) = < oo). 


Vn > 1. 


Suppose that <5 < 1. Then it follows from ([lo)) and Theorem 2.6 that £'^[P^+( 7 „ < oo)] = 1 


for all n > 1. That is, with probability one every excursion of the ERW to the right of the origin 
will eventually return to the origin. Similarly, if < 1 then all excursions to the left of the origin 
eventually return to the origin. Therefore, if 5, (^ < 1 then all excursions from the origin are finite 
and so the walk returns to the origin infinitely many times. Since all u}x{j) are uniformly bounded 
away from 0 and 1 this implies that the walk visits every site inhnitely often. 


If instead 5 > 1, then (10) and Theorem 2.6 imply that £',,[^^^+(71 = 00 )] > 0, and thus 
0 < £'r,[wo(l)]£r)[-Pa;+( 7 i = oo)] < R,(A„ > 1 , n > 1 ) < P,; ( lim = oo) , 
where the last inequality again follows from the fact that the 0 Jx{j) are uniformly bounded away 


from 0 and 1. Finally, we can conclude by Theorem 1.4 that 5 > 1 implies that the walk is 
transient to the right with probability 1. A similar argument shows that 5 > 1 implies that the 
walk is transient to the left. □ 


Proof of Theorem Since the limiting speed vq exists by Theorem |l.^|(ii)[ if the walk is recurrent 
then the speed must be uq = 0. Thus, we need only to consider the case where the walk is transient. 
First, assume that the walk is transient to the right (5 > 1). Since Tn/n = the existence 

of the limiting speed vq implies that lim„_,.oo Tn/n = 1/vq, where we use the convention 1/0 = oo. 
It is easy to see that = n + 2 Ylx<n Pn every n > 1 . Since the walk is transient to the right. 


( 11 ) 




x<0 


k=0 


1 Tn 

— = lim — 

Vq n^oo n 


= lim — n 
n^oo n \ 




n—1 


x<n 


= 1 + lim — 
n^oo Tl 




x=0 


and thus 
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2.5 


that n ^ Ylx=o '^n the same distribution as n ^ started with 


It follows from Lemma 
Vo = 0, and standard Markov chain arguments imply that 


lim 

J —^OO TJ / 


E, 


Vfl 


EZo' 


n^oo 77 , 


2=1 




Thus, we can conclude that 


E.. 


y,o 


Vo — 1 + 2 - 


V 1 

“ V- 
Z^i=0 


P^’°-a.s. 


-1 




Theorem 


2.7 


implies that the E^^la^] < oo (recall that <5 > 1 since the walk is transient to the 


right) and that ^ 14 ] < oo if and only if 6 > 2. Thus we can conclude that uq > 0 

6 > 2. Again, a symmetric argument for random walks that are transient to the left shows that 

uo < 0 6 > 2 . □ 


Proof of Theorem 1.8. The proofs of the limiting distributions in Theorem 1.8 relies on the connec¬ 


tion of the hitting times with the backward BLP Vi from Lemma 2.5 and the tail asymptotics 
for the backward BLP in Q . We will give a brief sketch of the general argument here and will give 
full details in the case <5 = 2 in Appendix We will refer the reader to previous papers for the 
details in all other cases. 

For the limiting distributions of the hitting times T„, recall that 

n—1 

(12) r,, = n + 2^P^ + 2^P4. 

a:=0 ai<0 

It follows from 0 that the third term on the right has a finite limit as n —)> oo, Prf-a.s., and 
therefore it is enough to prove a limiting distribution for the first two terms on the right side of 


(12). By Lemma |2.5| this is equivalent to proving a limiting distribution for + 14 under the 

measure P^'^. The proof of the limiting distribution for the partial sums of the BLP relies on the 
regeneration structure of the process 14 . Let denote the time of the k-th. return of the backward 
BLP to zero. That is, 

ro = 0 and = inf{i > rk-i : 14 = 0}. 

Also, let Wk = YliZ-Zi y- Note that {(r^ — rfc_i, 114:)}fc>i is an i.i.d. sequence under the measure 

P^’^ and that Theorem |2.7| implies that ri and Wi are in the domains of attraction of totally 
asymmetric stable distributions of index minjd, 2} and min{(5/2,2}, respectively. From this, the 
limiting distributions for n + 2 YZi=i standard. For the details of the arguments in cases 

(i) (v) we give the following references. 

• (5 G (1,2): See |BS08b| . pages 847-849. 

• 5 = 2: See Appendix jB) 

• (5 G (2,4]: See Section 9 in |KMllj . 

(7^ — 1 

• 5 > A: By (7) the second moment of the random variable Ylj=o yj finite. Thus, this 
is a classicaTcase covered_by the standard CLT for the Markov chain 17; see, for example, 
[Chu67[ 1.16, Theorem 


®For the second statement of (v) see also | KZ08| . proof of Theorem 3 and Section 6. The argument is based on 
the result due to A.-S. Sznitman |Szn001 Theorem 4.1] and gives the functional CLT for the position of the walk. 
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Finally, to obtain the limiting distributions for the position of the ERW from the limiting 
distributions of the hitting times T„, we use the fact that 

(13) {Tjn > n] <Z {Xn <m} C {Tm+r > n} U I inf Xk < m \ , 

for any m^n^r > 1. The key then is to control the probability of the last event on the right. To 
this end, it was shown in [Pet 121 Lemma 6.1] that the tail asymptotics for ri in Q imply that 


(14) 


Pr 


f inf Xk < mj < Cr^ \/m,r > 1 . 


Again, for the details of how to use (13) and (14) to obtain the limiting distributions for see 
the references given above. □ 


3. Mean and variance of the forward BLP 

Many of the calculations below are simplified using matrix notation. 

• 1 will denote a column vector of all ones, and p denotes a column vector with f-th entry 

p{i). 

• / is the identity matrix. 

• Dp will denote a diagonal matrix with i-th diagonal entry p{i). Similarly Di-p = I — Dp 
is the diagonal matrix with i-th diagonal entry 1 — p{i). 

• Recall that p = p ■ p, where p is the stationary distribution for the environment Markov 
chain. 

In this section and the next section, we will be concerned with a single increment of the BLP. Thus, 
we will only need to consider the environment at any fixed site x G Z. Therefore, in this section 
we will fix X and suppress the sub/super-script x for a less cumbersome notation. For instance, we 
will write Rj = for the Markov chain which generates the environment at x and the Bernoulli 
sequence is denoted {^ij)}j>i = {^xij)}j>i- 

3.1. Mean. 


Proposition 3.1. For every distribution rj onTZ 


lim 

n^oo 


=: A. 


^ ^ P 
n 1 — p 

Moreover, there exist constants C 5 , cg > 0 such that for any n G N and any distribution p on TZ 


(15) 
where 

(16) 


Ep’^iUi] -Xn-p-r\< 056 -'=®^, 


V 


r = r(p, K) = i I - K + 


(1 - p)pDi-pK 

1 — p 


-1 


p — Al. 


Proof. Recall that Sm is the number of successes before the m-th failure in the Bernoulli sequence 
^ = {^(i)}j>i) a-iid let Gm = Sm — be the number of successes between the (m — l)-st and 

the m-th failure in the Bernoulli sequence With this notation it follows from the construction of 
the forward BLP in Section [2] that 

n 

m=l 


( 17 ) 
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To compute Er\Gm] it helps to keep track of some additional information. Let Iq = Ri and for any 
m > 1 let Im be defined by 

Im = RSm+m+l- 

Note that Sm + m is the number of Bernoulli trials needed to obtain m failures. Therefore, Im = i 
if the next Bernoulli random variable after the m-th failure has success probability p{i). Since 

Pvi{^U)}j>S^+m+i G • I criCU), j <Sm + m+l))= Pi^{{Cij)}j>i G •), 
it follows that {Im}m>o is a Markov chain on the state space IZ. By the ellipticity assumption, 
p ; 7 ^ — >■ (0,1), the Markov chain {Im}m>o has the same unique closed irreducible subset IZq as 
the Markov chain {Rj}j>i- Therefore, {Im}m>o has a unique stationary distribution tt. While we 
did not assume any aperiodicity for {Rj}j>o, the ellipticity assumption implies that {Im}m>o is 
aperiodic, and since TZ is finite the convergence to stationarity is exponentially fast: if 11 is the 
matrix of transition probabilities for the Markov chain Im, then there exist constants C 7 ,C 8 > 0 
such that 

(18) sup llryll"'— 7 r||oo < £ 76 “^®”, Vn > 1, 

V 

where the supremum on the left is over all probability measures rj on IZ. 

Let g denote the column vector of length N with i-th entry g(i) = Ei[Gi]. Note that by the 
comparison with a geometric random variable with parameter pmax := maxjg 7 ^p(z) < 1 we see that 
g{i) < Pmax/(1 — Pmax) < oo for every i £lZ. Then, since Erj[Gm] = Er^[g{Im-i)] it follows from an 
ergodic theorem for finite state Markov chains that 

1 1 ” 

lim -Ell'^pi] = lim - ^ E^[Gm] = E^[g{Io)] = ^ 7 r(z)c/(z) = tt • g. 

n—¥oo Tl n—^00 Tl 

m=l iGTZ 

Therefore, to prove the first part of the lemma we need to show that vr • g = A. The following 
lemma accomplishes this task. It also provides useful information about {Im}m>o which we need 
in the rest of this section. 


Lemma 3.2. The sequence {Im}m>o is a Markov chain with transition probabilities given by the 
matrix 

n = (/ - DpK)-^Di_pK, 

and with a unique stationary distribution vr given by 

(19) TT = ~ = (1 + X)fi{I - DpK). 

I — p 

Moreover, 

(20) g = (/ — DpK)~^p and vr • g = A. 


Remark 3.3. In the first formula for vr in (19), the multiplicative factor 1/(1 — p) is needed for the 
entries of vr to sum to 1 since p,{I — DpK)! = piX — p) = 1 — p. The second formula for vr in (19) 
is equivalent since A = implies that = 1 + A. 


Proof. To compute the transition probabilities, for /c > 0 let be the N x N matrix with entries 

Mk{i,i') = Pi{Gi = k,h=i'). 

Obviously, Mo(i,i') = (1 —pi)Ki^ii = {Di-pK)iy and for A: > 1 by conditioning on the value of R 2 
we obtain that 

N 

= '^PiKi,eMk_i{i,i'). 

1=1 
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That is, Mq = Di-pK and = DpKMk_i for A: > 1. Combining these we get that = 
{DpK)^Di-pK. Since = Ylk>o we have that 

OO OO 

n = ^ Mfc = Y,iI^pK)^Di_pK = (/ - DpK)-^Di_pK. 
k=0 k=0 

(Note that since DpK is a matrix with non-negative entries and with z-th row sum equal to p{i) < 1, 
the Perron-Frobenius theorem implies that all eigenvalues of DpK have absolute value strictly less 
than one, and thus I — DpK is invertible.) It it is easy to check that vrll = vr by noting that pK = p 
and, thus. 


(21) pDi_pK = fi{I - Dp)K = p{I - DpK). 

Finally, we give a formula for g. For A: > 1 one easily sees that 

DiGi >k)=Y^ >k-l). 

he'll 

Iterating this, we obtain 

(22) P,(Gi >k)= • • • (p(4_i)K.,_„4) = ei{DpK)% 

where in the last equality we use the notation e* for the row vector with a one in the A-th coordinate 
and zeros elsewhere. Therefore, 


g{i) = Ei[Gi] = Y, PiiGi >k) = Y e^{DpK)H = e^{I - DpK)-^DpKl = ei{I - DpK)-^p 


k>l 


k>l 


and we get (20) as claimed. From this and the formula for vr in (19) it follows immediately that 
TT • g = A. □ 


In the critical case p = 1/2, Lemma 3.2 and (21) give the following simpler formula for the 
stationary distribution vr that will be useful below. 


Corollary 3.4. If p = 1/2 then tt = 2p{I — DpK) = 2iJ,Di-pK. 

Thus far we have proved the first part of Proposition |3.1[ Next we show the existence of a vector 
r such that (15) holds. To this end, for any n > 1 let r„ = (rn(l),r,i(2),... ,r„(N'))* be the column 
vector with i-th entry 


rnii) = eYPi] - An = ^ {Ei[G^] - A) 


771=1 


Then 


71—1 


r„ = ^ (n™ ^g - Ivr • g) = ^ (n™ - Ivr) g, 

771=1 771=0 

where in the last equality the matrix Ivr is the N x N matrix with all rows equal to the vector vr 
which is the stationary distribution for {Im}m>o- It follows from (18) that the entries of 11"* — Ivr 
decrease exponentially in m, so that the sum in the last line converges as n —)> oo. That is, 

OO 

(23) lim r„ = r := ^ (11"* - Ivr) g. 

Moreover, since Ep'‘^[Ui] — Xn = p ■ r„, then 

\eY[Ui] - Xn - p ■ r\ = \p ■ {rn - r)\ < Y 11^^”* - 7r|| 


771=0 




1 — 6 “*^® 


771=71 
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where the last inequality follows from (18). 

Finally, we give an explicit formula for the vector r. To this end, since vrll = vr it follows that 
(l7r)n = Itt = n(l7r) and thus (rh” — Ivr) = (11 — Itt)™ for all m > 1. Therefore, 

OO OO 

(24) r = (/ - l7r)g + ^ (11 - Itt)™ g = ^ (II - Ivr)™ g - (vr • g)l = (/ - 11 + l7r)'^g - Al. 


m=l 


m=0 


Substituting the expressions for tt, II, and g and simplifying we obtain (16). 


□ 


In closing this subsection, we note the following Corollary which will be of use later. 

Corollary 3.5. tt • r = 0. 

Proof. Since tt is the stationary distribution for the Markov chain with transition matrix II it follows 
that 7r(/ — n + Itt) = tt, or equivalently tt{I — II + Ivr)”^ = vr. Using the formula for r in (24), it 
follows that vr • r = 7r{I — II + l7r)“^g — A'7r-l = 7r-g — A = 0, where the last equality follows from 
( 2 §. 

□ 


3.2. Variance. The main result of this subsection is the following proposition. 
Proposition 3.6. There exists a constant cg > 0 such that for any distribution r] on TZ 

Var^(C/i| Uo = n) , ^ cg 
n n 

where the constant v is given by the formula 


(25) z. = z.(p, K) = Var^(Gi) + 2 Cov^(Gi, Gi+fc) = (1 + A)(A + 2fiDpKr), 

k=l 

and r is the vector from Proposition \3.1\ In particular, if p = 1/2 then 

(26) p = 2 + ApiDpKr = 4/iiApiF(I — K + 2(1 — p)^iAi_piF)“^p. 

Proof. First note that for any measure rj on TZ, 

( n \ n 

=Y.yarp{Gk) + 2 Y CoVr,{Gk,Ge). 

k=l / k=\ l<k<£.<n 

Let Un = Var 7 r(C/i| Uq = n) = Yaxj^ (Z]fc=i Gk)- The proof will consist of three steps. 

Step 1. Show that \vn — Var^(C/i| Uq = n)\ is bounded by a constant uniformly in n and rj. 
Step 2. Prove that there is a constant C > 0 such that — nl < G/n. 

Step 3. Calculate v explicitly and show that ( [^ and ( [^ hold. 

Step 1. For any fc > 0 and i G 7^ let Vk{i) = and 

Vfc = {vk{l),Vk{2),.. .,Vk{N))K 

With this notation, we have that 

(27) ^8Xp{Gk) = - (r/^'^■^g)^ VA: > 1, 

and 

(28) Cov^(Gfc, Gt) = plT-^i-k - (r?n^-ig)(??n^-ig), VI < < 1 

Note that in the special case where rj has distribution vr these become 
(29) \aiT,{Gk) = vr(vo - Ag), and Cov^(Gfc, Gi) = 7r(v£_fc - Ag), for 1 < /c < £, 
since vrll = vr and vr • g = A. The following lemma is elementary. 
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Lemma 3.7. sup^>o ||vfc||oo < oo. 

Proof. First of all, note that the Cauchy-Schwartz inequality implies that 

Ei[GiGi+k] < E^[Gl]Ei[Gl^^] < maxFl^G?] = ||vo||oo, 

and thus it is enough to prove that Ei[Gi] < oo for every i £ TZ. The last inequality is obvious by 
comparison with a geometric random variable with parameter pmax = maxjgTj p(i) <1. □ 

Lemma 3.8. There exist constants, cio,cii > 0 so that for any distribution rj on TZ and any 
l<k<i 

(30) |Var^(Gfc) - Var^(Gfc)| < cioe"'""*' and |Cov^(Gfc, Gi) - Cov^(Gfc, Ge)\ < 


Proof. The key observation is the exponential convergence to the stationary distribution vr for the 
In {j 


Markov chain {Im}m>o as noted in (18) above. From this, it follows easily that 
TT-vol < C 7 ||vo||ie-'=«(^-^) and 


Vo 


g - T • g| < C7||g||ie 


—cs{k—l) 


The first inequality in (30) then follows easily from the above bounds and the representations for 
and (29) taking into account the fact that is always a probability 


the variances in 

distribution so that \rjU.^~^g, + vr • g| < 2||g 


To obtain the bound on the difference of the covariance terms in (30), note that the representa¬ 


tions in (28) and (29) imply that 

|Cov^(Gfc,G^)-Cov^(Gfc,G,)| < 

< 


rjli^ - (t/II'' ^g)(T/n^ ^g) - vr(v£_fe - Ag) 


Tfc-l 


(r/n^ ^ - 7r)(v£_fe - Ag) 


+ 




A - r/n^“^g 


< cje 


—Cs{k—1) I 


Vi_k - Aglli -k ||g||oo||g||iC7e 


-cs{e-i) 


where the last inequality again follows from (18) and the fact that vr • g = A. Therefore, it will be 
enough to show that there exist constants GjC' > 0 such that 

(31) ||vfc - Ag||i < Cllgllfe"*^'^, VA: > 1. 

To this end, note that by conditioning on (Gi, Ji) we get that 

OO 

Ei[GiGk+i] - \Ei[Gi\ = Y.Y. =n,h = i') {E,\Gk\ - A). 

I'eTl n=0 

By (18) maxjgT^ \Ei[Gk] — A| = maxjg-;^ |ein*^“^g — tt • g| < C 7 ||g||ie“'^®^^“^\ and thus it follows that 

\Ei[GiGk+i] - XE,[Gi]\ < Ei[Gi] (c7||g||ie-'=«('=-i)) . 

Since this gives a bound on each of the entries of — Ag, the inequality in (31) follows. □ 


To complete step 1 in the proof of Proposition |3.6| we notice that Lemma 3.8 implies 


\nn - Var^(17i| Uq = n)\ = 


Var^ 


Y.Ok 


— Var. 


V 




\k=l 


\k=l 


< ^ cioe 
k=l 


-Cllfc 


-k 2 ^ Cloe 

l<k<i<n 


-ciii 


Note that the sums on the right are uniformly bounded in n and that the constants cio, cn do not 
depend on r]. Thus, we conclude that 


(32) 


sup 

n,r] 


Var. 


-VarJ^G, 


\k=l 


\k=l 


< OO. 
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Step 2. Note that 
1 


n n 


n—l 


J^Var^(Gfc) + - ^ Cov^(Gfc,G,)=Var^(Gi) + 2j^h --jCov^(Gi,Gi+fc), 

k=l l<k<i<n k=l ^ 

where in the last equality the change in the indices is due to the fact that vr is the stationary 
distribution for the Markov chain {Im}m>o- Therefore, 

oo CO n—l j 

Var^(Gi) + 2 ^ Cov,(Gi, Gi+fc) - ^ = 2 ^ Cov.(Gi, Gi+k) + 2 ^ Cov,(G'i, Gi+fc). 


k=l 

It follows from (|29l) and 


k=n 


k=l 


that 


|Cov^(Gi,Gi+fc)| = Itt • (vfc - Ag)| < Ce 


-C'k 


Therefore, for some C" > 0 

OO 

Var7j-(Gi) + 2 Cov7i-(Gi, Gi+a:) 


k=l 


n 


n—l 


_ o ^ 

< 2 Y Ce~^'^ + - Y Cke~^'^ < —. 

n n 

k=n k=l 


Step 3. Since Ei[G‘i\ = E^i(2fc “ ^)Pi{Gi > k), recalling the formula in (22) for Pi{Gi > k) 
we obtain 


(33) 


VO = ^(2A: - l){DpKfl = (I + DpK){I - DpK)-^DpKl 

k=l 

= {I- DpK)-\l + DpK){I - DpK)-^ip = {I- DpK)-\l + DpK)g. 


Using (33) and (29), we have that 

Var^(Gi) = vr ((/ - DpK)-\l + DpK) - A/) g = 7r(/ - DpK)-\{l - A)/ + (1 + X)DpK)g 

= (1 - X)7r{I - DpK)-^g + (1 + X)T^DpK{I - DpK)-^g. 
Since vr = (1 + X)pi{I — DpK) this simplifies to 

(34) Var,(Gi) = (1 - X^)i,g + (1 + XfpiDpKg. 

To compute Cov 7 r(Gi, Gi+fc) we need a formula for v^. Recall that = Pi{Gi = n,/i = 

i'), so that by conditioning on Gi and Ii we obtain 

OO OO 

E^[GiGi+k] = Y,^Mn{i,i')Ei>[Gk] = (iU^R^-^g 

n=0 n=0 

Using this and the fact that M„ = {DpK)^Di-pK we get 

OO / OO \ 

Vfc = (M^n'^-ig) = Di-pKn^~^g 


n=0 


\n=0 / 

= DpK{I - DpK)-^Di_pKll^-^g. 


Now, note that if in this equation the matrix ^ is replaced by Ivr (recall this is the matrix with 
all rows equal to vr) then since vrg = A we have 


DpK{I - DpK)-‘^Di-pKl-Kg = XDpK{I - DpK)-'^Di-pKl = XDpK{I - DpK)-‘^{l - p) 

= X{I - DpK)-^DpK{I - DpK)-^{l - p) = X{I - DpK)-^DpKl 
= X{I - DpK)-^p = Ag. 
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Therefore, we can re-write the formula for Cov 7 r(Gi, Gi+fc) in (29) as 


Cov^(Gi, Gi+fc) = vr • (vfc - Ag) = 7rDpK{I - DpK)-^Di_pK ( n""' - Ivr ) g. 


jk—l 


Recalling the definition of r in (23), this implies that 

oo 

Cov^(Gi, Gi+fe) = 7 TDpK{I - DpK)-^Di_pKv 

k=l 

(35) = 7 r(/ - DpK)-^DpK{I - DpK)-^Di_pKT = (1 + X)^^DpKIlr, 

where the last equality follows from Lemma |3.2[ 


Combining (34) and (35), we obtain that 

= (1 - A^)/xg + (1 + XfiiDpK^ + 2(1 \)iJiDpKI{r. 


To further simplify this, note that it follows from (24) and Corollary 3.5 that 

g = (I — n + l7r)(r -|- Al) = (I — n)r + Al. 

Re-arranging this we get Hr = r -|- Al — g, and putting this back into the above formula for v we 
get 


V = {\ — A^)/xg -|- (1 -|- X)^nDpKg -|- 2(1 -|- X)nDpK[r Al — g) 

= (1 — A^)/xg -|- (A^ — l)/jDpKg -|- 2(1 -|- X)/jDpKr 2(1 -|- X)XfiDpK\ 

= (1 — A^)/x(/ — DpK)g 2(1 -|- X)ijDpKr 2(1 -|- X)Xp 
= (1 — A^)/x • p -|- 2(1 -|- X)fiDpKr + 2(1 -|- X)Xp = (1 -|- A)A -|- 2(1 -|- X)fj,DpKr 


where in the second to last equality we used the formula ( 20 ) for g, and in the last equality we 
used that ^ ■ p = p = . 


□ 


4. The backward BLP and parameter relationships 


Throughout this section we will assume that we are in the critical case p = 1/2. We shall discuss 
the backward BLP, give explicit formulas for parameters 5 and 5, and derive the key relationship 
between them. 

Consider first the backward BLP V. To obtain the results about V from those for the forward 
BLP U 

• we need to replace p with 1 — p everywhere in order to switch from counting “successes” 
to counting “failures”; 

• we have to account for the fact that V has one “immigrant” in each generation: recall 

that Vi = while Ui = i G N. 

The above observations lead to the following statements whose proofs are identical to those for 
U. We shall state the results only for the critical case, since we use V solely in the critical setting. 


Proposition 4.1. Let p = 1/2. For every distribution rj on TZ 

lin, ^ ^ 

n^oo Tl 

Moreover, there exist constants ci 2 , C 13 > 0 such that for any n G N and any distribution rj on TZ 

\Ep[Vi \Po = n] - n - (1 + r/ • f)| < 


where 


Y = y{1 - p,K) = {I - K + 2ppDpK) ^ (1 - p) - 1. 
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Proposition 4.2. Let p = Xj^l. There exists a constant C 14 > 0 such that for any distribution rj 
on TZ 


Var^(Pi| Po = n) 


— V 


n 


< 


Cl4 

n 


where the constant v is given by the formula 

(36) = p(l — p, K) = 2 + 4,pDi-pKr = 4p,Di-pK{I — K + 2ppDpK)~^{l — p) 

For reader’s convenience we list the relevant parameters for U and V side by side. 



TT = 2/i(l — DpK) = 2pDi-pK-, 

TT = 2p{l — Di-pK) = 2pLDpK-, 

(37) 

r = {I - K + 2{1- p)pDi_pK)~^p - 1; 

v = {I-K + 2ppDpK)-\l - p) - 1 


u = 2 + ApDpKr = 2 + 27fr; 

V = 2 + ApDi-pKr = 2 + 27rf. 

In the 

above formulas, recall that 



K is the transition matrix for the Markov chain Rj used to generate the cookie environment 
at each site. The row vector p is the stationary distribution for this Markov chain, 
p is the column vector with Tth entry p{i), 1 is a column vector of all ones, and Dp and 
Di-p are diagonal matrices with Tth entry p{i) and 1 — p{i), respectively. 

The row vectors vr and if give the limiting distribution of the next “cookie” to be used 
after the re-th failure or success, respectively, in the sequence of Bernoulli trials ^(j) at a 
site. 

The column vector r and the parameter u are related to the asymptotics of the mean 
and variance of the forward BLP U as given in Propositions 3.1 and|3.6[ and f and u are 


related to the asymptotics of the backward BLP V in the same way by Propositions 4.1 
and 14.21 

Now we can define the parameters 5 and 6 that appear in Theorems 


1.6 


1.8 


(38) 


5 = 


2 ri ■ r 


(5 = 


2 r] ■ f 


n v 

Note that the parameter 5 can be computed in terms of p, K and rj. If we wish to make this 
dependence explicit we will write S = S(p,K,p). In particular, with this notation we have that 
5 = (i(l - p,K,g). 

We close this section with a justification of the statement of Theorem 


1.6 


that 5 + (5 < 1. 


Proposition 4.3. v = v and 6 + 6 = 1 — 2/n. 

Proof. We shall need the following lemma. 

Lemma 4.4. r + f = (/i(r + f))l. In particular, r + r is a constant multiple of 1. 


Let us postpone the proof of Lemma 4.4 and continue with the proof of Proposition 4.3 Recall 
that by Corollary |3.5|7r • r = 0. Similarly, ff • f = 0. Therefore, Lemma |4.4| and the formulas for n 


and V in (37) imply that 


= 2 + 27r(r + f) ^'^^2 + 2fj.{r + f) ^'^^2 + 27f(r + r) = n. 


Moreover, from the above line we see that p{r + f) = vI2 — 1. Combining this with Lemma 4.4 


we 


get that r + f = (i^/2 — 1)1. Then by (38) we have 5 + 5 = 2ry-(r + f)/i/ = l — 2/i/ as claimed. □ 


Proof of Lemma 4-4 ■ Recall that r„ is the column vector with f-th entry 

rn{i) = -n = Ei[Sn] 


— n. 
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where Sn is the number of successes before the n-th failure in the sequence of Bernoulli trials 
{?(j)}i>i- Lst ^U) be the number of successes in the first n Bernoulli trials. If = k, 

then there are n — k failures among the first n Bernoulli trials and so Sn is equal to k plus the 
number of successes before the k-th failure in shifted Bernoulli sequence Therefore, 

by conditioning on and Rn+i (the type of the (n + l)-st cookie), we obtain that 

n 

= E E Pi{Zn = k, Rn+1 = i') (k + Ei> [S'fc] - n) 

k=0 

n 

= EE Pi{Zn = k, Rn +1 = i') {2k + rk{i') - n) 

i'GlZ k=0 

n 

= Ei [2Zn — n] + EE Pi{Zn — fe, Rn-\-l — i ) 

i'GTt k=0 

n n 

= Y.^,[2p(R,)-l\+e+-r+Y.T. Pi{Zn — k, Rn+1 — i )(rfc(i ) T{i )). 

j=l i'£TZ k=0 

Since ||rfc — r||oo decreases exponentially in k and lim„_ 5 .oo = k, Rn+i = i') = 0 for every 

k > 0 and i' G TZ, it follows from the dominated convergence theorem that the last sum vanishes 
as n —>■ oo. Since r„ —)• r as n —)• oo, we have shown that 


(39) r{i) = lim { VE* [2p{Rj) - 1] + aK^r > , Vi G 7^. 

[j=i j 

where the limit on the right necessarily converges. Similarly, a symmetric argument interchanging 
the roles of failures and successes yields 


(40) 


f(i) = lim ^ V Ei [1 - 2p(RP] + eiK^r > , Vi G 7^. 
n^oo I ^ ^ I 


Since clearly = re for all re, it follows from ( |39[ ) and (40) that 

n 


r + f = lim /V”(r + r) = lim — K^{r + f) = (l/u)(r + f). 

n._^r>n n^OO Ti 


J = 1 


(Note that the second limit above is needed since we did not assume that the Markov chain Rj is 
aperiodic on the closed irreducible subset TZq.) □ 


5. The non-critical case 


In this section we consider the non-critical case p = p-j) ^ 1/2 and give the proof of Theorem 1.5 
The proof will be obtained through an analysis of the forward and backward BLP from Section [2 


Theorems 2.6 and 2.7 which were the basis for the proofs of Theorems 1.6 1.8, cannot be directly 


applied in the non-critical case p ^ 1 /2. The main idea in this section will be to construct a coupling 
of the non-critical forward/backward BLP with a corresponding critical forward/backward BLP and 
then use this coupling and the connection of the BLP with the ERW to obtain the conclusions of 
Theorem 11.51 
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5.1. Coupling. Let iL be a transition probability matrix for a Markov chain on 7^ = {1, 2,... ,N} 
with some initial distribution rj. We will assume that this Markov chain satishes the assumptions 
stated at the beginning of Section 1.1. In particular, it has a unique stationary distribution /r 
supported on a closed irreducible set TZq C TZ. Suppose that we are given two functions po,pi : 
TZ —)■ (0,1) such that pi > po. Let pi := {pi{l),pi{2),...,pi{N)), i = 0,1, and assume that 
/r • Pi > /i • po = 1/2. 

Next, we expand the state space for the Markov chain to be 7^ = {1, 2,..., 2iV}, and for any 
e G (0,1] we let be a 2N x 2N transition matrix and p be a column vector of length 2N given 
by 


(41) 


Ke = 


il-e)K 

eK 

On 

K 


and p = 


Pi 

Po 


where 0^ denotes an N x N matrix of all zeros. It is clear that the unique stationary distribution 
for TLg is given hy fi = (0, p), where 0 is a row vector of N zeros and p is the stationary distribution 
for the transition matrix K. Moreover, p ■ f) = // • po = 1/2 so that and p correspond to a 
critical ERW. 

Let and be the associated forward and backward BLP for the critical ERW corresponding 
to p, iLg, and with initial distribution = ((1 —e)r^, ep). Let U and V be the forward and backward 
BLP for the non-critical case corresponding to pi, K, and the initial distribution p. We claim that 
for any s G (0,1] these BLPs can be coupled so that 

• Uk > for all k > 0; 

• Vk< for all k>0. 


To this end, recall that is the distribution for the i.i.d. family of Markov chains {Rj}j>o, x G Z, 
with transition matrix K and initial condition p. We can enlarge the probability space so that 
the measure also includes an i.i.d. sequence {j^jxez of Geometric(e) random variables (that is 
Fr)(j^ = k) = (1 — e)^e for A: > 0) that is also independent of the family of Markov chains {Rj}j>i, 
X G Z. We let tOxij) = PiiRj) and also define a different cookie environment = {Ld^(j)}x£Z,j>i 
by 

i PiiRj) 3<le 

\poiRj ) j > le- 


OJ, 


:U) = 


It is clear from this construction that the cookie environments to and w are coupled so that 
w|(j) < uixU) for all X G Z and j > 1. Given such a pair of coupled cookie environments, 

let {ixij)}x&'L, j>i and {txij)}xez,j>i be families of independent Bernoulli random variables with 
^xiJ) ~ Bernoulh(t<; 3 ;(j)) and CxU) ~ Bernoulli(£h^(j)) that are coupled to have C^(j) < ^x(j) for 
all X G Z and j > 1- If t7 and V are the forward and backward BLP constructed from the Bernoulli 
family {^xij)}x,j and and are the forward and backward BLP constructed from the Bernoulli 
family {^^(j)}a;,j, then the couplings Uk > tjf, and 14 < ^k all fe > 0 follow immediately. More¬ 
over, it is easy to see that U, V, and W have the required marginal distributions under this 
coupling. 

As noted above, the forward and backward BLP and are critical BLP to which Theorems 
|2.6| and |2.7| apply. In the applications of these theorems, however, one must replace the parameter 
6 = 6{p,K,p)hy 

4 = 4(pi, po, K, p) := (5(p, ke,pe). 


The following lemma shows, in particular, that the parameter 5^ can be made arbitrarily large by 
letting e —)■ 0. 
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Lemma 5.1. Let e G (0,1], r/ he an arbitrary distribution on TZ, and and be the BLPs 
constructed above. Then 


(42) 


<^£(Pi, Po, K, rj) = (5(po, K,t]) + ' 


4^- Er=i(l-£)'^'-HPi-Po 


i^(po,i^) 

In particular, 5e{pi,'Po,K,r]) is a continuous strictly decreasing function of e on (0,1] which equals 
6 {po,K,r]) when e = 1 and satisfies 


(43) 


lim 6e{pi,Po,K,r]) = oo. 


Proof. It follows from the formula for 6 given in (38) that 

^e{pi,Po, K,'n) = -^4-where fg = r(p,it'£) and = i'{p,Ke 


Therefore, if fe = (r£(l), re(2),... ,f£{2N)Y then (42) will follow of we show that for all e G (0,1] 
(44) i>£ = i/(p, Ks) = u{po, K), and 


(45) 


= r{p,ke) = [r{po,K) + 2'^{l-eyK^ ^(pi - po), r(po, iiT) 

i=i 


Note that (45) states, in part, that the last N coordinates of are equal to r(po,iir) for any 
e G (0,1]. This follows easily from the fact that the bottom right N x N block of the transition 
matrix in (41) is equal to K. Similarly, since the states i = 1,2,... ,A^ are transient for the 
Markov chain with transition matrix K^, it follows that the stationary distribution tt corresponding 
to the pair (p,!^^) is given by 7r(p, i^^) = (0,7r(po, ill)). Thus, the formula for the parameter n in 
(37) implies that 

i>e = 2 + 27r{p,ke) • fe = 2 + 27f(po,7ir) • r(po,77) = i^(po,i^). 


This proves (44), and it remains to show (45) for the first N coordinates of r^. 
Applying the formula for r{i) from ( |39[ ), we have that 

I [2 (Pi(^i)l{i<7.} +4>o(^i)l{j>7,}) - l] + eik^rA 
“ I =1 J 


(46) 


J = 
n 


lim \ ^ 2(1 - eYaK^ ^(pi - po) + [2po(-Ri) - 1] + eiK^Ve 

71 .—I • ^ ^ 

i=i i=i 


It follows from the form of the matrix and the fact that the last N entries of the vector r^ are 
equal to r(po, K') that 


lim 

n-^OQ 



eiA:’'r(po, A') 


= 0 . 


Together with (|39|) this implies that the limit of the last two terms in (46) is the i-th entry of the 


vector r(po, K). Since the limit of the first term in (46) is the corresponding infinite sum we obtain 


Finally, to prove (43), it is enough to show that 


(47) 


^i?A'^(Po - Pi) = oo. 
i=o 


Recalling that pi > po and /i • pi > • Po = 1/2 we conclude that there is a strict inequality 

Pi{i) > Po{i) for some state i G IZq which is therefore a recurrent state for the Markov chain Rj. 
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Since the i-th entry of equals the expected number of times the Markov chain Rj visits 

■ □ 

Without loss of generality we may assume that p > 1/2. We shall 


the state i, (47) follows 


5.2. Proof of Theorem 1.5 

apply Lemma 


5.1 


Choose Pi = p. Since p > 1/2, there exists po = {pq{1),pq{2), ... ,po{N)Y G 


(0,1)-^ such that 

• Po{i) < Piii) for all i G TZ. 

• p-po = 1/2. 

Let the distribution r/ on 77 be hxed. 

Po, K, rf) > 4, and we will keep this choice of e f ixed for the remainder of the proof. 

that = oo with positive probability 


By Lemma 5.1 we may choose an e G (0,1] so that 5^ = 


Transience: Since Jg > 1, it follows from Theorem 


2.6 


for any initial condition Uq = y > 1 for the forward BLP . Since the above coupling of the 
forward BLP is such that Uk > for all /c > 1, this implies that = oo) > 0 for any y > 1. 

From this, the proof that the ERW is transient to the right is the same as in the proof of Theorem 

Ol 

Ballisticity and Gaussian limits 

Z^i=o 

the above coupling is such that Vk for all /c > 0 this implies that 


Since 5^ > 4, it follows from Theorem 


2.7 


that 


and 


Vf have finite second moments when the backward BLP W is started with 1// = 0. Since 


(48) 


E'Y'^[{aYY] < oo and E, 


y,o 


El 

i=Q 


< OO. 


As noted in the proof of Theorem |1.7[ a formula for the limiting speed vq when the ERW is transient 
can be given by 


E, 


v,o 


— = 1 + 2 




E^’V] 


Since (48) implies that the fraction on the right is finite, we can conclude that the limiting speed 


uo > 0. The proof of the Gaussian limiting distributions for the ERW is the same as for the proof 
of the limiting distributions of the critical ERWs in the case <5 > 4 since all that is needed are the 


finite second moments for the backward BLP given in (48) 


6. Proofs of Theorems O AND o 


We shall discuss the proof of Theorem |2.7[ since Theorem 2.6 can be derived in exactly the 
same way. The main idea behind the proof of Theorem |2.7| is very simple. The rescaled critical 
backward BLP can be approximated (see Lemma 6.1 below) by a constant multiple of a squared 
Bessel process of the generalized dimension 


4(l + ??-r) 4 ^ Lpl2(i _ 

P V 

The squared Bessel process of dimension 2 is just the square of the distance of the standard planar 
Brownian motion from the origin, and this dimension is critical. For dimensions less than 2 the 
squared Bessel process hits 0 with probability 1. For dimensions 2 or higher the origin is not 
attainable. Thus, 6 = 0 should be a critical value for V. This suggests that the process dies out 
with probability 1 if 5 > 0 and has a positive probability of survival if <5 < 0. Moreover, the tail 
decay exponents for (^ > 0 can also be read off from those of the corresponding squared Bessel 
process. The boundary case 5 = 0 is delicate and has to be handled separately. 
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Nevertheless, a lot of work needs to be done to turn the above idea into a proof. This is 
accomplished in |KMllj and |DK15j for the model with bounded cookie stacks. The proofs of the 
respective results in the cited above papers can be repeated verbatim provided that we reprove 
several lemmas which depend on the specifics of the backward BLP V. Therefore, below we only 
provide a rough sketch of the proof and state several lemmas which depend on the properties of 
our V. These lemmas cover all <5 G M and are used later in the section to discuss the three cases: 
5 > 0, (5 = 0, and 5 < 0. Their proofs are given in the next section. 


6.1. Sketch of the proof. Let us start with the already mentioned approximation by the squared 
Bessel process]^ 

Lemma 6.1 (Diffusion approximation, Lemma 3.1 of |KM11| . Lemma 3.4 of |DK15| L Fix an 
arbitrary e > 0, y > e, and a sequence yn ^ y as n —)■ oo. Define = 


Vi 


[ntj Acte 

n 


t > 0. 


Then, under the process converges in the Skorokhod (Ji) topology to Y{- A aj) where 

Y is the solution of 

(50) dY{t) = {l + ri-r)dt + ^/p^dW{t), Y{0)=y. 

Remark. For the ERW with bounded cookie stacks the convergence is known to hold up to Uq (or 
in D{[0,oo)) if cjg^ = oo) as long as the corresponding squared Bessel process has dimension other 
than 2 (see |KZ14l Theorem 3.4] for the forward branching process). But such result does not seem 
to significantly shorten the proof of Theorem ] 2. 7[ and we shall not show it. 

To prove Q we need some tools to handle V when it starts with y <C en or falls below en. The 
idea again comes from the properties of Y. It is easy to check that Y^ for <5 7 ^ 0 (and InE for 
5 = 0) is a local martingale. Then by a standard calculation we get that for all a > 1 and y G N 

The same power (logarithm for 5 = 0) of the rescaled process V is close to a martingale, and we 
can prove a similar result for all G M (for <5 = 0 it is sufficient to set a = 2 below). 

Lemma 6.2 (Exit distribution. Lemma 5.2 of [KMlI], Lemma A.3 of [DKIS]). Let a > 1, |x — a-^ | < 
and 7 be the exit time from Then for all sufficiently large y G N 

A 


< a^-i) - 


+ I 


< a-P^. 


One of the estimates needed for the proof of Lemma 6.2 is the following lemma which shows that 
the process V exits the interval not too far below or above afi 

Lemma 6.3 (“Overshoot”, Lemma 5.1 of |KM11| L There are constants ci 5 ,ci 6 > 0 and N € N 
such that for all x > N, y > 0, and every initial distribution rj of the environment Markov chain 

max PP^{Vr,„ > X + y\Tx < cTo) < ci 5 

0<z<x \ / 


and 


max <x-y)< cise 

x<z<ix ' 


“Following the number of each lemma in this subsection are the corresponding results in the literature which it 


replaces. 
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Lemma 6.3 shows that the process V typically exits the interval close enough to the 

boundary so that we can repeatedly apply Lemma 6.2 to couple V with a birth-and-death-like 
process to obtain the following estimate on exit probabilities from large intervals (a^, a“) and 
ultimately handle V when it is below en. 

Lemma 6.4 (Lemma 5.3 of [KM 11] and Lemma A.l of |DK15] i. For each a G (1,2] there is an 
io G N and a small positive number A such that if i,m,u,x G N satisfy £o < i < m < u and 
|x — a™| < then 

h~{m) — h~{i) 


where for j > 1 


h (u) — h {i) 


h^{j) = 


< pV,x/„V 
— V 


> 'Tan) < 


[a =F a ^ 


h'^{m) — h'^{tj 
h+{u) - h+{i) ’ 

if6>0; 
if6<0-, 
if6 = 0. 


Moreover, for <5 7 ^ 0 there are Ki 
j >i 


N 


nl=i 

> ( 0 , 00 ), i = 1 , 2 , such that Ki{£) — )■ 1 as £ ^ 00 and for all 


(51) 




Proof. The proof is essentially identical to the proof of [KMlll Lemma 5.3] and relies only on the 
properties proved already in Lemmas 6.2, 6.3 and the fact that the BLP are naturally monotone 
with respect to their initial value Vq. The proof of [KMlll Lemma 5.3] was given for J > 0, but 
essentially the same proof holds for <5 < 0 with only minor changes needed due to the form of the 
functions h^{j). □ 

The proofs of Lemmas 6.1-6.3 will be given in Section]^ We close this section with a brief 
discussion of how to finish the proof of Theorem 2.7 using these lemmas in the cases 5 > 0, = 0 

and 5 < 0. 

6.2. Case <5 > 0. Substituting the lemmas from the above subsection for the corresponding lemmas 
in [KMllj and repeating the proof given in this paper we obtain Q for 5 > 0, which, in particular, 
implies that P^’^{o'q < 00 ) = 1 for all <5 > 0. 

6.3. Case 5 = 0. The proof of Q for 5 = 0 is identical to the proof of [DK151 Theorem 2.1]. All the 
ingredients which depend on the specifics of V are contained in the lemmas above. Note again that 
while the 2-dimensional squared Bessel process never hits zero, ([^ implies that P^’^{a^ < 00 ) = 1 
for 5 = 0. 

6.4. Case 5 < 0. All we need to show is that P^’^{aQ = 00 ) > 0 for every y > 0. Notice that by 
the strong Markov property and monotonicity of the BLP with respect to the starting point for 
every m > i > max{^o, log 2 (y -|- 1)} 


= 00 ) > Ppy{a^ = 00 , 


V 


<^o)>P 


K,2” 

V 


{aX = oo)pPy{T^ru < ) 

> = oo)P^’y{TY^ < aX). 


By the lower bound of Lemma 6.4 and (51) we can choose and fix sufficiently large m > £ so that 

> 0 . 


dL2"*/ V \ T dV,2" 

P ’ = 00 ) = hm Pf' 


{a^i > T^u) 




h- 


[m] 


h-{£) 


Moreover, by the ellipticity of the environment, P^’^{T 2 m < a^) > 0, and we conclude that 
= 00 ) > 0 for every y >0. 
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7. Proofs of Lemmas I6.1H6.3I 


We shall prove a more general diffusion approximation result and then derive from it Lemma 6.1 
Lemma 7.1 (Abstract lemma). Let b € M., D > 0, and Y{t), t>0, be a solution of 
(52) dY{t) = bdt + y^DY{t)+dW{t), Y{0) = x>0, 

where W{t), t > 0, is the standard Brownian motioi^ Let integer-valued Markov ehains := 
{Zn^k)k>o satisfy the following conditions: 

(i) there is a sequence G N, Nn —)• oo, Nn = o{n) as n ^ oo, f : N 
as X ^ oo, and 5 : N —)■ [0, 00 ), g{x) \ 0 as x —)■ 00 such that 

(E) \E{Zn,i - Zn,o I Zn,o = m) - b\ < fim V Wi); 

Var(Z„,i I Znfi = m) 


[ 0 ,oo), f{x) 0 



m V Nr, 


- D 


< g{m V Nn 


max 


l<k<[Tn)/\- 


{Zn,k - Zn,k-lf = o{n^) 


as n 


00 , 


where = inf{A; > 0 : Zn^k > x}. 

Set Znfi = \nxn\, Xn ^ X as n ^ 00 , and Yn{t) = t > 0. Then Yn ^ Y as n ^ 00 , 

where ^ denotes convergence in distribution with respect to the Skorokhod (Ji) topology. 

Proof. The proof is based on |EK861 Theorem 4.1, p. 354]. We need to check the well-posedness of 
the martingale problem for 


A = 




on (^^[OjCX)) and conditions (4.1)-(4.7) of that theorem. The well-posedness of the martingale 
problem follows from |EK 86 [ Corollary 3.4, p. 295] and the fact that the existence and distributional 
uniqueness hold for solutions of (52) with arbitrary initial distributionsp] Thus we only need to 
check the conditions of the theorem in |EK 86 ] . 

To this end, define processes An(t') and Bn{t) by 

^ \rit\ ^ \nt\ 

■— 9 / YcXcifZnk \ Zn,k—l), d3n(t') .— \ E(^Zn k Znk—1 \ Znk—l)- 

k=l k=l 

It is elementary to check that the processes Mn{t) := Yn{t) — Bn{t) and Mn{t) — An{t), t > 0, are 
martingales for all n G N. Therefore, it is sufficient to check the following five conditions for any 

'^Let us remark that X{t) ~ 4y{t)ID satisfies dX{t) = {Ab/D)dt + X{t)+dB{t) and, thus, is a squared Bessel 
process of the generalized dimensi on 4 b/D. 

more detailed discussion of (521 can be found immediately following (3.1) in | KZ14| . 




















30 

T,r > 0. 

(53) 

(54) 

(55) 

(56) 

(57) 
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lim E 

n^oo 


lim E 

n^oo 


lim E 

n^oo 


sup \Yn{t) - Yn{t-)\'^ 

_t<TATr" 

sup \Bn{t) - Bn{t-)\ 

_t<TATr^ 

sup \An{t) - An{t-)\ 


= 0 . 


= 0 . 


= 0 . 


lim sup \Bn{t) — bt\ = 0, P-a.s. 


t<TAT> 


lim sup 
t<rAT> 


An{t)-D / Y+{s)ds 


= 0, P-a.s. 


Note, in the above conditions that = infjt > 0 : Yn{t) > r} = jn. 

Condition (53) is simply a restatement of condition (ii) in the statement of Lemma 6.1 Condi¬ 
tions (54) and (55) follow from conditions [(^E) and|(i)KV), respectively. Indeed, 


lim E 

sup Pn(t) - Pn(t-)P 

1 

= lim —^E 

max | 

71^00 

t<TATr" 


l^k<{Tn)ATrn 






and similarly 


lim E 

sup \An{t) - Anit-)\ 

= lim —^E 

max Var(Z„,fc | Zn,k-i) 

n^oo 

IKTAtA 

n—¥oo 

l<fc<(rn)VT,^" 


“’C lim I"’'’ = 0. 

n^oo 

where in the last inequality we used that Z^^k-i < rn for k < and that Nn < rn for n large 
enough. To check condition (56), note that 

161 1 ^ 

sup \Bn{t) -bt\ < — + sup - \E [Zn,j - ^nj-1 | “ b\ 

t<TATr" ^ l<k<(Tn)AT^" ^ j=\ 

(Tn)AT^" 


n n 


1=1 


n 


sup f{m) 

mS[Af„,rn] 
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Since this final upper bound is deterministic and vanishes as n —>• oo, we have shown that condition 

[tn\ 


(56) holds. Finally, to check condition (57) note that 


\Anit) - D / Y^{s) ds 


\tn\ 


\tn\ 


^2 ^2 I Zn,k-l) ^2 ^2 ^n,k-l 


k=l 


k=l 


-"u- 

n 


n 


■^n, \tn\ 


1 

< ^ 


^ ^ I ^n,k—l) DZn^k—l 


l<k< [tn\: 

^n,k — l^^n 


1 

H-2 


'[vai'(2n,fc I 2n,fc-l) + 




l<fc< [tnj: 


n 


[tnj 


n 


I 

n, [mj 


(58) 


< ^ 


g{^n)Zn,k-l + 


tAlni‘^D + g{Nn)) D 


l</i;< \tn \: 


n 


+ -{t- 
n 


[tn\ 


n 


Z 


n, [tn\ ’ 


where the last inequality follows from condition |(i)[ V) and the assumption that g is non-increasing. 
Now if t < (equivalently, tn < then Zn^k-i < fn for all k < [tnj, and so the first sum in 


, if t < r/"- then whereas if 


(58) is at most rtn^g{Nn). As for the last term in 
t = then tn is an integer and the last term in (58) is zero. Thus, we conclude that 

An{t)-D /V+(s)ds 


sup 


/o 


, , TNJ2D + g(Nn)) Dr 

< rTg{Nn) + - ^ ” +-^ 0. 

n n n^oo 


This completes the proof of condition (57) and thus also the proof of the lemma. 


□ 


Proof of Lemma 6.1, The diffusion approximation is obtained from Lemma 7.1 in two steps: (1) 
construct a modihed process V for which it is easy to check the conditions of Lemma 7.1 and 
conclude the convergence to the diffusion for all times; (2) couple the original process to the modihed 
one so that they coincide up to the hrst time the processes enter (—oo, iV„]. Since = o(n), this 
gives the diffusion approximation up to the hrst entrance time to (—oo,ne] for every e > 0 as 
claimed. 

Note that the backward BLP I 4 can be written as I 4 = Y2rn=2^ ^mi where — is 

the number of failures between the (m — l)-st and m-th success in the sequence of Bernoulli trials 
{?a;(i)}i>i- It follows that 

Vfc-i+i Li-i+i 

(59) Vu= Y. Gi = 14_i + 1+ (g^-i), k>l. 

m=l m=l 

We now construct the family of modihed processes Vn = {W,fc}fc>o as follows. Fix any sequence 
Nn G N such that 2 < Nn 00 and Nn = o{n) as n —)• 00 . For all sufficiently large n (we want 
Nn <C ny„) set 

(fn,fe-l + l)VAf„ 


kn,o — ) and 1 T 1 T 


E 

i=i 


(G"-l), k>l. 


Note that the modihed process Vn is naturally coupled with the BLP V since we use the same 
sequence of Bernoulli trials in the construction of both. In particular, if we start with Vn^ = 
Ynyn\ > en> Nn then the two process are identical up u ntil exiting [en, 00 ). 

It remains now to check the conditions of Lemma 7.1 for the family of modihed processes Vn- 
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Parameters and condition (i). By Propositions 4.1 and 4.2, condition (i)(E) holds for Zn = Vn with 
b = 1 + T] ■ r and f{x) = ci 2 e~'^^^^, and condition (i)(V) holds with D = v and g{x) = C 14 /X. 
Condition (ii). Fix T,r > 0. We need to show that 


lim 

n^oo Ti^ 



(Li.fe-l + liVWn 

2 

max 

1+ (6/-1) 


l^k<{Tn)ATXn 

i=i 



= 0 , 


where = inf{/i; : Vn^k > rn}. To see this, note that if n is large enough so that Nn < rn then 


2 


max 

.\<k<{Tn)/\Tr- 


(^n,fc-l + l)VA^n 

E (Gj -1) 

j=i 


— 2 


max max (& — 1 ) 

l<k<(Tn) Nn<m<rn+1 I •' 


i=i 


1 

< ^ 




max max 


y=0 


l<k<(Tn) Nn<m<rn-\-l \ 
i 7 = 1 


- 1 ) 


> y 


.3/2 


E A |E(G‘-l)>v^ 


y>{rn)^/^ \ J—1 

Finally we apply Lemma |A.l to get that the expression in the last line does not exceed 
r-3/2 


n 

3/2 


+ rT C ( exp <1 -C' 

y>{rn)^/^ 


y 


Texpl-C'^y}^ 


ytyy ( 8 rn) 

S ^ ( A/V (8.2/3)) } + - » 

y>{rn)'^/^ 


as n ^ oc. 


By Lemma 7.1 we conclude that the processes t i—)• with initial conditions = [nyn] 

and jjn ^ y converge in distribution as re —>■ oo to the process Y defined by (50). As noted above, 
since Vn can be coupled with V until time rj^, this gives us the desired result. □ 


Next we give the proof of the overshoot estimates in Lemma 6.3 since they are needed for the 


proof of Lemma 6.2 


Proof of Lemma 6.3. The proof follows closely the arguments in the proof of Lemma 5.1 in |KMllj . 
We begin by noting that 

T3V,z(^r ^ < \ V ^ V\ ^ P^'^{Vi>x + y) 

max P„ ’ (Vt-v > X + y < an ) < max - ^ -. 

0<z<x ^ ^ ^ - 0<z<x p^iVi > x) 

Since the (averaged) distribution of the sequence {Gj}j>i dehned above doesn’t depend on x G Z 
we will let {Gj}j>i denote a sequence with this same distribution. It follows from (59) that 

■'2 + 1 / 

jTn \ z, -r ± -r / 

yV,znr ^ ^ I „,i ^ 


Pri (z + 1 + Xfcl i^j “1) > X + y 

max P^’^iV^v >x + y\T^ <(Xo)< max - - -——- 

Pjz + l + EYi^Gj-l)>x 


Pr 


[E-ZT{G, -l)>y + 


(60) 


= max 

0<m<x p 
■V n 


m 


(E-ZTiG, - 1 ) > rre 
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where the last equality follows from the substitution m = x — z — 1. We therefore need to give 
a lower bound on the denominator and an upper bound on the numerator of (60). For the lower 
bound on the denominator we will use the following lemma. 


Lemma 7.2. 


inf min Pi j — 1) > 0 j >0. 


n>l i£TZ 




Proof. The event “ 1) ^ 0} occurs when there are at least n failures by the time of 

the n-th success in the sequence of Bernoulli trials. This occurs if and only if there are at least n 
failures in the first 2n — 1 trials, or equivalently less than n successes in the first 2n — 1 trials. That 
is, 

« -1) >oj = p. = p. (y: («^')-^) s“) ■ 

The last probability converges to 1/2 for each i G P by Lemma |A.4[ □ 

Next, we show how to use Lemma [7.2| to obtain a lower bound on the denominator on the right 


side of (60). For any m > 1 let pm = min{n > 1 : Y21=ii^j ~ 1) ^ Then, clearly 


A I E(Gj-i)> 

i=i 


m = 


X] I X] -'^)>m,pm = n 

j = ^ 


n=l 


(61) 


> ^ F/y { (Gj - 1) > 0, Pm = n \ > CPnipm <x-m), 

\j=n+l 


n=l 


where the constant C > 0 in the last line comes from Lemma 1 7.2 
For the numerator in (60), 


P, Y.(Gj-l)>y + 


m 


1=1 


- I -^) > m + pm = n 


n=l 

x—m 

sE 

n=l 




+ Prj I {Gj 1) > Pm — n 

n=l \j=n+l 


iGlZ i<m 


max max Pj (Gi — l>m — t'H— Gi — l>m — ^)> Prj{pm = n) 


+ PviPm = 'll) ^ max max Pi I y^iGj - 1) > 

^ I iG/Z k<x » 


n=l 





(62) 


< 


Vvm 


+ maxmaxPj '^{Gj - 1) > ^ > PJpm <x- m), 

i&'R, k<x \ ^ I ‘ 


where Pmax = xaaxi^'jip{i) < 1. Applying (61) and (62) to (60), and using the concentration bounds 
in Lemma A.l finishes the proof of the first part of Lemma 6.3 The proof of the second inequality 

□ 


in Lemma 


6.3 is essentially the same and is therefore omitted. 
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Proof of Lemma 6.2. As noted prior to the statement of Lemma |6.2[ if Y{t) is the process in 
that arises as the scaling limit of the backward BLP, then Y°{t) (or logy(t) when 5 = 0) is 
a martingale. The proof of Lemma 6.2 is then accomplished by showing that (or log 14 when 
(5 = 0) is nearly a martingale prior to exiting the interval aj). The proof follows essentially the 

same approach as in [KMlli proof of Lemma 5.2(ii), pp. 598-599]. To this end, let s G (^“([O, oo)) 
have compact support, and satisfy 

<5 


s(x) = 


2 3a 


X" (5/0 

log X (5 = 0 

Define 'Hk '■= (y{Vi : i < k). Then, a Taylor expansion for s(x) yields that on the event {7 > k} 


for all X G I —, — 
3a 2 


?V,x 


(¥) 




nk 


a2 


[Vk+i-Vk\nk] + 


" (Yk 

2a^ 


[{Vk+ 1 -Vkf\nk] +ej,k, 


where the term ej^k comes from the error in the Taylor expansion and can thus be bounded by 


(63) 


£j,k Y 


1^ Hop t^V,x 

6a3i ^ 


(Vk+i - VkY 


-Hk 


3/4 


< 


I s'" 11 

I* ||CXD 

6a3i 


(Aff )3/4 < (7a-3j72^ 


where the second-to-last inequality follows from the fourth moment bound in Lemma A.3 and the 
last inequality follows from the fact that I 4 < on the event {7 > k}. Next, it follows from 


Propositions 4.1 and |4.2| that on the event {7 > k} 

\E^’^ [Vk+i - 14| - (1 + I? • f)| < ci 2 e-‘=«^'= < ci 2 e- 


(64) 
and 

(65) 


\E^^^ [(Pfc+i - Vkf\ Uk] -LVk\< C. 


Combining (|63|), 

( 66 ) 


^V,x 


m 


and (65) we see that 
-Hk 


Vk 


1 


a^ 


Vk 




a^ 


yVk 


2 aJ 


where on the event {7 > fe} the error term Rj^k is such that 

(67) \Rj^k\ < Ca-V/'^ + Ca-^e-^'^' + Ca-^^ < C", 


Now, it follows from (49) and the fact that s(x) = x^ in [1/a, a] that 

{I + r] ■ r)s'{x) + ^xs"{x) = ^ {(1 “ 6)s'{x) -b xs"(x)} =0, Vx G [1/a, a]. 

Therefore, the quantity inside the braces in ( 66 ) vanishes on the event {7 > k} and so we can 
conclude that — YY}k=o'^ ^ ^ martingale with respect to the filtration Rn- From this 

point, the remainder of the proof is the same as that of Lemma 5.2 in [KMllj . We will give a 


sketch and refer the reader to |KM11| for details. First, the diffusion approximation in Lemma 6.1 
can be used to show that E^’^Yl] < Ca^ and thus it follows from the optional stopping theorem 
and (67) that 

7-1 


( 68 ) 


TpV,X 


V, 

of 


— ■s{x/a^) 


< £j'-' 


E 


k=0 


< Ca-V^. 
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Next, the over/under-shoot estimates in Lemma 6.3 are sufficient to show that 


(69) 



C/D 

V 

\ad J 




< Ca-P^. 


Finally, since |x — a^\ < a?P^, it follows that l'S(^) — 1| < Ca P^. From this and the estimates in 
(68)-(69) the conclusion of Lemma 6.2 follows. □ 

Appendix A. Appendix 

In this appendix we prove several technical estimates which are used in the analysis of the BLP 
in Section We begin with the following concentration bound. 

Lemma A.l. Let p = 1/2. There exist eonstants C, C' > 0 and yo < oo such that for all n > 1, 
y > 2 / 0 ; o,'’T'd for any initial distribution p for the environment Markov chain we have 


Pr. 


EA - 

i=i 


> y 


< C (expi-C' 




y 

y V 8n 


+ exp{-C'y}) 


Before giving the proof of Lemma A.l we note that a similar concentration bound is true for 
sums of i.i.d. random variables with finite exponential moments. 

Lemma A.2 (Theorem 111.15 in |Pet75j ). Let li, 12, ■ • • be a sequence of i.i.d. non-negative random 
variables with E[Yi] = y and E[ePP] < oo for some Aq > 0. Then, there exists a constant C > 0 
such that 

/ ” \ c 

yn 


P 


k=l 


> y < exp -C 


y V n 


Proof of Lemma ] A. 1\ Let i* be any positive recurrent state, so that /u(i*) > 0. Fix such an i* and 
set Jo = inf{j > 1 : Rj = P}, and for /c > 1 let J^ = inf{j > Jk-i '■ Rj = **}• Also, for A: > 1 let 

A —1 

E 

j=Jk-l 

Note that {lfc}fc>i is an i.i.d. sequence with Erj[Yi] = Eri[Ji — Jo]/2 = l/(2/i(P)). 

Now, for any y > 0 and any integers n,N>l 

/ 2n-\-y 


P, EA-1)>!/ =A E«;< 


n 


vi=i 


\ 1=1 


N 


< Pr, (jo > if)+Pri [Jn - Jo > 2n + |j + < 

Now, let N = [/i(i*) (I + 2n)J so that 


n 


\k=l 


2n+| > AP[Ji-Jo] + | and n < AP[yi] - | 
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Therefore, for y > 0 sufficiently large (so that y/8—l/(2/r(i*)) > y/9; thus y > 36/^(i*) is sufficient) 
we have 

Pv ( - 1) > y ) < Pr, (jo > I) + [Jn -Jo> NE[Ji - Jo] + I) 




N 


+ Pr, Y."Pk<NE[Yi] 


\k=l 


9 


Since the cookie stack {wo(j)}i>i is a finite state Markov chain and i* is in the unique irreducible 
subset, then it follows easily that there exists constants Ci, (72 > 0 such that 


PjJoPiz] <Cie 


-C2y 


for all y > 0 and for all distributions rj of the environment Markov chain. From this it also follows 
that Jk+i — Jk has exponential tails, and since Yk < Jk+i — Jk then Yk also has exponential tails. 
Therefore, we can apply Lemma |A.2 to get that 

Pr, (Jn -Jo> NE[Ji - Jo] + I) + Yk < NE[Y,] - 


-Cs 


-C4 


-C5-, 


<g NV(y/4) g ^4iVV{!;/9) < 2e ^ (9V)Va , 

Now, recalling the definition of N (which depends on both n and y) we see that N < y(i*){8n V 
y)/2 < ( 8 n V y)f2. Therefore, we can conclude that 

Pri - 1) > yj ^ Cie-^^y + 

for all n > 1 all y > \36/y{i*)] =: yo- 

The proof of the upper bound for the left tails is similar. Indeed, 




n-y 






If y > n then the probability is 0 and there is nothing to prove. Assume that n > y. Then the last 
probability is equal to 

^2{n-y)+y-l 


2n—y—l 


pA E (1-^0< 


n-y \ = 


Pv\ E (1 -?*)<(«- y)- 


i=i 


i=i 


This probability can be handled in exactly the same way as the upper bound for the right tails by 
setting n' = n — y. Then in the exponent we shall have (for y < n) —y‘^/{8{n — y)\/y) < —y‘^/{8n) = 
—y'^/{8n\/y). □ 


The concentration bounds in Lemma A.l can be used to prove the following moment bound for 
sums of the (7,-. 
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Lemma A.3. Let p = Xj^l. Then there is a constant A = A{yQ,C,C') > 0 (see Lemma A.l) such 
that for any initial distribution y on the environment Markov chain and all n > 1 

' n \ 4 

Etc- - 1 ) 


Er, 


\ 2 = 1 


< An^. 


Proof. We shall use the fact that for a non-negative integer-valued random variable X 


E{x^)<i + cY,y''Pix >y)- 

y=l 


Set A = 


Er=i(G.-l) 


Then by Lemma 


i/o-i 


A.l 


for all n > yo /8 


OO 


E{X^) < 1 + C ^ y^P{X >y)+ C” ^ y3^g-C'y2/(8nVy) ^ ^-Cy^ 


y=l 


8n 


y=yo 
' 0,2 ; 


< Ki{yo, C, C) + C" Y, + 2C" Y 


y^e 


y=yo 

8n 


y=yo 


< 


K2{yo,C,C') + C”Y 


3^-C'yy{8n) 


y e 


y=yo 


Since the function y^e ^’’P'P is directly Riemann integrable, we can bound the sum in the last 
line by rif times a Riemann sum approximation. That is, 


8 n 


^ y3g-C'yV(8n) <r?Y 
y=0 


y=yo 


“ ^ y V 1 

« 1 g 8 y^j _ 

n / aAi 


poo 

~ dy, as n —> oo. 

Jo 


□ 


Lemma A.4. Assume that p = 1/2. There exists a constant v > 0 such that for any initial 
distribution y, under the measure Prj we have that Er=i(?* “1/2) =► vZ, where Z is a standard 
normal random variable. 


Proof. As in the proof of Lemma |A. 1 [ fix a recurrent state i* £ TZ and let Jq, Ji, J 2 , • • • be the 
successive return times of the Markov chain to i*. Set 

Jk—^ 

Zk-= Y fe-l/2). ^>1- 

j=Jk-l 


{Zk)k>i is an i.i.d. sequence of centered square integrable random variables. Let Ln 
and Ln = max{A; G N : Jk-i < n} otherwise. Then 


1 


\/n 


n Ln 

Y{c^-l/2)-YZk 


i=l 


k=l 



Jo—1 

1 / 2 ) 


2=1 


-^n +1 1 

E (^*- 1 / 2 ) 

2 =n+l 


1 




max (Jfc - Jk-i) 

l</c<n+l 


0 if Jo > n 
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Since Jq is almost surely finite and {J^ — Jk-i}k>i is i.i.d. with finite second moment, it follows 
that the last expression above converges to 0 in probability. Therefore, it remains only to prove 
that 


1 -^n 

rEz* 

/n 

k=l 


vZ 


for some v > 0. By the law of large numbers, L„/n —)■ ^-s-j and the desired 

conclusion follows from |Gut091 Theorem I.3.1(ii), p.l7] with = iJ,{i*)Eri[Zf]. □ 

Appendix B. Details of the limiting distribution in the case 6 = 2 
In this appendix we will give the details of the proof of the limiting distributions in Theorem 


1.8 in the case 6 = 2. This is the most subtle case in Theorem 1.8 and has not been fully written 


down for ERWs even for the model with boundedly many cookies per site. Even though the proof 
is very similar to that for one-dimensional random walk in random environment for the analogous 
case f [KKS75[ pp. 166-168]), some of the details are different due to the fact that the regeneration 
times Tj have infinite second moment. 

First, recall the definition Q of the a-stable distribution Lafi and the fact that if Z has distri¬ 
bution with a 7 ^ 1, then cZ has distribution La^bc'^ for any c > 0. However, if a = 1 then 
a re-centering is needed to get a distribution of the same form. This difference is indicative of 
the fact that for a ^ 1 there is a natural choice of the centering for totally asymmetric a-stable 
distributions: the distributions La^b have mean zero when a > 1 and have support equal to [ 0 , oo) 
when a < 1. In contrast, when a = 1 there is no canonical choice of the centering for the totally 
asymmetric 1-stable distributions. To this end, for 6 > 0 and ^ G M let Li^b,i be the probability 
distribution with characteristic exponent given by 

log f Li^b,^{dx) = iu^ — b\u\ ( 1 + ^ log 1^1 sign(tt) 


vr 


That is, the distribution differs from Li b by a simple spatial translation. 

Recall that is the time of the k-th. return o f the backward BLP V) to zero and that Wk = 
^ 1^- When 5 = 2, it follows from Theorem 

(70) 


2.7 


that 


P^’^{ri > t) ~ Cat ^ and > t) ~ c^t as t —> oo. 


for some constants C 3 = 03 ( 0 , r?) > 0 and C 4 = 04 ( 0 ,r]) > 0. Let m{t) = £',^’'^[lTiln;i/i<t}] be the 
truncated first moment of Wi (note that the above tail asymptotics of Wi imply that m{t) ~ 
C 4 logt). Then, it follows from |DurlO[ Theorem 3.7.2] or |Fel71l Theorem 17.5.3] that there exist 
constants 6 ' > 0 and G M such that 


(71) 


lim R 


,u,o (Y2=1 Wk -nm{n) ^ \ . . 

n 1 - - - <x \ - Li^b’,i’[.x), 


X G 


and that there exists a constant A > 0 such that 


(72) 


lim rT’O 


r-n — nr 


A^/n\o^ 


< x] = $(x) 


X G 


Since (72) implies that is typically close to n, we wish to approximate the distribution of 

n~^ b) by that of n~^ Wk- Indeed, we claim that the difference converges to zero in 
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P^’^-probability. To see this, note that for any e > 0, 


pVfl 

V 


n n/r 

i=l k=l 


> en 


< 

— V 


\n/r\ 


r \ — n\ > ri 


3/4 


+ P. 


.y,o 


= P, 


y,o 


(^KPJ -n\> +Pr 


Vfl 

V 


^ Wk>en 

^2[n3/'‘J+3 

^ Wk>en 


k=l 


It follows from ( [tT] ) and ( |72[ ) that both terms in the last line above vanish as n —)• cx) for any e > 0. 
Therefore, we can conclude from this, the limiting distribution in ( |71[ ), and the fact that has 
the same limiting distribution as n + 2 ^ 


lim P, 


y,o 




r„-2fm(f) 


n 


< X 1 = lim R 


y,o^ Si=i ^ 


n^oo 
[n/fj 


V 


< X 


n 


= lim R 


v,o ( Wk - ^m(^) ^ {x - l)f 


V 


= L 


1 , 6 'R 


(x—l)r'^ _ 


-^ 1 , 6 ,^ (^); 


n/r 2 

where in the last equality we have b = ^ and ^ = 1 + ^ log(p). This completes the proof 

of the limiting distribution for when S = 2 with 6 > 0 as above, D{n) = ^ + im(p), and 
a = f/{2ci). 

Before proving the limiting distribution for Xn, we first need to remark on the specific choice of 
the centering term D{n) in the limiting distribution for T^- One cannot use an arbitrary centering 
term growing asymptotically like o“^logn, but the above choice of the centering term by the 
function D{t) = ^ + iR[lTil{iyi<t/r}] grows regularly enough due to the tail asymptotics of ITi so 
that 


(73) 


lim D[mn) — D{n) = 0, if rUn ^ n as n ^ oo. 


For t > 0 let T{t) = inf{s > 0 : sD{s) > t}. Then, it follows that 

at 


(74) 


m 


log(t) 


and T{t)D{T{t)) = t + o{T{t)), as t —)■ oo. 


The first asymptotic expression in (74) follows easily from the definition of T{t) and the fact that 
a“^logt. For the second asymptotic expression in 


D{t) a ^logt. For the second asymptotic expression in ( |74[ ), note that for s sufficiently large 
sD{s) is strictly increasing and right continuous in s. Moreover, if sD{s) is discontinuous at sq 
then the size of the jump discontinuity is 2(so/7)^R(VFi = sq/t). Therefore, 


|r(t)R(F(t))-t| <2 


m 


R ( IFi = 

r 


since if sD{s) is continuous at s = T(t) then the above difference is zero while if sD{s) is discon¬ 
tinuous at s = r(t) then the difference is at most the size of the jump discontinuity at that point. 
Since the tail asymptotics of VFi imply that xR(lTi = x) = o(l), the second asymptotic expression 
in (74) follows. 
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Now, for any x G M and n > 1 let nin^x ■= rr(n) + (iog^) 2 l V 0. Since mn,x grows asymptotically 
like r(n) as n —)• oo it follows that 

n - mn,xD{mn,x) _ n - mn,j;-P(r(n)) 


rur, 


m. 


n,x 


n — 


= lim ■ 

n^oo 


(r(n) + (4V) "(rt")) , 

= lim “ 

+ (logn)^ + (logn)^ 


j^DiVin)) _ 


i 2 ’ 


where the first equality follows from (73) and the last two equalities follow from (74). Similarly, 


letting Mn^x = + [\/r] it follows that 


(76) 


lim 

n^oo 


n - Mn,xD{Mn,x) _ -X 


M, 


n,x 


,2 ■ 


Finally, it follows from (13) and (14) that 


D I '^rnrL,x TXln,xD(jnn^x) ^ n 'Triji^xDljnn^x) 

I > 


rur 


rur 


= Pv{Tm^,x >n)<Pr, <x)< P,{Tmx.,x > n) + 0{n-P^) 

\n/[logn)^ ) 

PMn,x ~ P^n,xD{PIn,x) ^ n — Mn^xD{Pl^n,x)'\^ 0 (n~^P) 


= Pr, 


From the limiting distribution for Tn, together with (75) and (76), we conclude that the first and 
the last probabilities in the display above converge to 1 — x/a^). 


References 

[AB016] Gideon Amir, Noam Berger, and Tal Orenshtein. Zero-one law for directional transience of one dimen¬ 
sional excited random walks. Ann. Inst. Henri Poincare Probab. Stat., 52(l):47-57, 2016. 

[A016] Gideon Amir and Tal Orenshtein. Excited Mob. Stochastic Processes and their Applications, 126(2):439- 
469, 2016. 

[BSOSa] Anne-Laure Basdevant and Arvind Singh. On the speed of a cookie random walk. Probab. Theory Related 
Fields, 141(3-4):625-645, 2008. 

[BS08b] Anne-Laure Basdevant and Arvind Singh. Rate of growth of a transient cookie random walk. Electron. J. 
Probab., 13:no. 26, 811-851, 2008. 

[BW03] Itai Benjamin! and David B. Wilson. Excited random walk. Electron. Comm. Probab., 8:86-92 (electronic), 
2003. 

[Chu67] Kai Lai Chung. Markov chains with stationary transition probabilities. Second edition. Die Grundlehren 
der mathematischen Wissenschaften, Band 104. Springer-Verlag New York, Inc., New York, 1967. 

[DK12] Dmitry Dolgopyat and Elena Kosygina. Scaling limits of recurrent excited random walks on integers. 
Electron. Commun. Probab., 17:no. 35, 14, 2012. 

[DK15] Dmitry Dolgopyat and Elena Kosygina. Excursions and occupation times of critical excited random walks. 

AREA Lat. Am. J. Probab. Math. Stat., 12(l):427-450, 2015. 

[Dolll] Dmitry Dolgopyat. Central limit theorem for excited random walk in the recurrent regime. AREA Rat. 
Am. J. Probab. Math. Stat., 8:259-268, 2011. 

[DurlO] Rick Durrett. Probability: theory and examples. Cambridge Series in Statistical and Probabilistic Mathe¬ 
matics. Cambridge University Press, Cambridge, fourth edition, 2010. 

[EK86] Stewart N. Ethier and Thomas G. Kurtz. Markov processes. Wiley Series in Probability and Mathe¬ 
matical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Inc., New York, 1986. 
Gharacterization and convergence. 

[EKM^OO] Nathalie Eisenbaum, Haya Kaspi, Michael B. Marcus, Jay Rosen, and Zhan Shi. A Ray-Knight theorem 
for symmetric Markov processes. Ann. Probab., 28(4): 1781-1796, 2000. 
























EXCITED RANDOM WALKS 


41 


[Fel51] 

[Fel71] 

[Gut09] 

[Har52] 

[Holl5] 

[HS12] 

[Jag75] 

[Jif69] 

[KKS75] 

[KMll] 

[Kni63] 

[KOS14] 

[KurSl] 

[KZ08] 

[KZ13] 

[KZ14] 

[MPV06] 

[MR06] 

[Pet75] 

[Pet 12] 

[Petl3] 

[Pet 15] 

[PinlO] 

[PT16] 

[SznOO] 

[Sznl2] 

[T6t95] 

[T6t96] 


William Feller. Diffusion processes in genetics. In Proceedings of the Second Berkeley Symposium on 
Mathematical Statistics and Probability, 1950, pages 227-246. University of California Press, Berkeley 
and Los Angeles, 1951. 

William Feller. An introduction to probability theory and its applications. Vol. II. Second edition. John 
Wiley & Sons, Inc., New York-London-Sydney, 1971. 

Allan Gut. Stopped random walks. Springer Series in Operations Research and Financial Engineering. 
Springer, New York, second edition, 2009. Limit theorems and applications. 

T. E. Harris. First passage and recurrence distributions. Trans. Amer. Math. Soc., 73:471-486, 1952. 
Mark Holmes. On strict monotonicity of the speed for excited random walks in one dimension. Electron. 
Commun. Probab., 20:no. 41, 7, 2015. 

Mark Holmes and Thomas S. Salisbury. A combinatorial result with applications to self-interacting random 
walks. J. Combin. Theory Ser. A, 119(2):460-475, 2012. 

Peter Jagers. Branching processes with biologieal applications. Wiley-Interscience [John Wiley & Sons], 
London-New York-Sydney, 1975. Wiley Series in Probability and Mathematical Statistics—Applied Prob¬ 
ability and Statistics. 

Miloslav Jifina. On Feller’s branching diffusion processes. Casopis Pest. Mat., 94:84-90, 107, 1969. 

H. Kesten, M. V. Kozlov, and F. Spitzer. A limit law for random walk in a random environment. Com- 
positio Math., 30:145-168, 1975. 

Elena Kosygina and Thomas Mountford. Limit laws of transient excited random walks on integers. Ann. 
Inst. Henri Poincare Probab. Stat., 47(2):575-600, 2011. 

F. B. Knight. Random walks and a sojourn density process of Brownian motion. Trans. Amer. Math. 
Soc., 109:56-86, 1963. 

Gady Kozma, Tal Orenshtein, and Igor Shinkar. Excited random walk with periodic cookies, December 
2014. To appear in Ann. Inst. Henri Poincare Probab. Stat. 

Thomas G. Kurtz. Approximation of population processes, volume 36 of CBMS-NSF Regional Conference 
Series in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 
Pa., 1981. 

Elena Kosygina and Martin P. W. Zerner. Positively and negatively excited random walks on integers, 
with branching processes. Electron. J. Probab., 13:no. 64, 1952-1979, 2008. 

Elena Kosygina and Martin Zerner. Excited random walks: results, methods, open problems. Bull. Inst. 
Math. Acad. Sin. (N.S.), 8(1):105-157, 2013. 

Elena Kosygina and Martin P. W. Zerner. Excursions of excited random walks on integers. Electron. J. 
Probab., 19:no. 25, 25, 2014. 

Thomas Mountford, Leandro P. R. Pimentel, and Glauco Valle. On the speed of the one-dimensional 
excited random walk in the transient regime. ALEA Lat. Am. J. Probab. Math. Stat., 2:279-296, 2006. 
Michael B. Marcus and Jay Rosen. Markov processes, Gaussian processes, and local times, volume 100 of 
Cambridge Studies in Advanced Mathematics. Gambridge University Press, Gambridge, 2006. 

V. V. Petrov. Sums of independent random variables. Springer-Verlag, New York-Heidelberg, 1975. Trans¬ 
lated from the Russian by A. A. Brown, Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 82. 
Jonathon Peterson. Large deviations and slowdown asymptotics for one-dimensional excited random 
walks. Electron. J. Probab., 17:no. 48, 24, 2012. 

Jonathon Peterson. Strict monotonicity properties in one-dimensional excited random walks. Markov 
Process. Related Fields, 19(4):721-734, 2013. 

Jonathon Peterson. Extreme slowdowns for one-dimensional excited random walks. Stochastic Processes 
and their Applications, 125(2):458-481, 2015. 

Ross G. Pinsky. Transience/recurrence and the speed of a one-dimensional random walk in a “have your 
cookie and eat it” environment. Ann. Inst. Henri Poincare Probab. Stat., 46(4):949-964, 2010. 

Ross G. Pinsky and Nicholas F. Travers. Transience, recurrence and the speed of a random walk in a 
site-based feedback environment. Probab. Theory Related Fields, pages 1-62, 2016. 

Alain-Sol Sznitman. Slowdown estimates and central limit theorem for random walks in random environ¬ 
ment. J. Eur. Math. Soc. (JEMS), 2(2):93-143, 2000. 

Alain-Sol Sznitman. Topics in occupation times and Gaussian free fields. Zurich Lectures in Advanced 
Mathematics. European Mathematical Society (EMS), Zurich, 2012. 

Balint Toth. The “true” self-avoiding walk with bond repulsion on Z: limit theorems. Ann. Probab., 
23(4):1523-1556, 1995. 

Balint Toth. Generalized Ray-Knight theory and limit theorems for self-interacting random walks on Z^. 
Ann. Probab., 24(3):1324-1367, 1996. 



42 


ELENA KOSYGINA AND JONATHON PETERSON 


[T6t99] Balint Toth. Self-interacting random motions—a survey. In Random walks (Budapest, 1998), volume 9 of 
Bolyai Soc. Math. Stud., pages 349-384. Janos Bolyai Math. Soc., Budapest, 1999. 

[Zer05] Martin P. W. Zerner. Multi-excited random walks on integers. Probab. Theory Related Fields, 133(1):98- 
122, 2005. 

Elena Kosygina, One Bernard Baruch Way, Department of Mathematics, Box B6-230, Baruch Col¬ 
lege, New York, NY 10010, USA 

E-mail address: elena.kosygina@baruch.cuny.edu 

URL: WWW.baruch.cuny.edu/math/f acultystaff/ekosygina/kosygina.html 

Jonathon Peterson, Purdue University, Department of Mathematics, 150 N University Street, 
West Lafayette, IN 47907, USA 

E-mail address: peterson@purdue.edu 
URL: WWW. math. purdue. edu/~pet erson 



